-- Logs begin at Fri 2022-07-15 18:38:10 UTC, end at Fri 2022-07-15 19:55:20 UTC. -- Jul 15 19:13:39.119205 dv-cinder-799871-34-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 15 19:13:39.128012 dv-cinder-799871-34-iscsi devstack@keystone.service[80254]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 15 19:13:39.130056 dv-cinder-799871-34-iscsi devstack@keystone.service[80254]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Jul 15 19:13:39 2022] *** Jul 15 19:13:39.130056 dv-cinder-799871-34-iscsi devstack@keystone.service[80254]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 15 19:13:39.130056 dv-cinder-799871-34-iscsi devstack@keystone.service[80254]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 15 19:13:39.130056 dv-cinder-799871-34-iscsi devstack@keystone.service[80254]: nodename: dv-cinder-799871-34-iscsi Jul 15 19:13:39.130056 dv-cinder-799871-34-iscsi devstack@keystone.service[80254]: machine: x86_64 Jul 15 19:13:39.130056 dv-cinder-799871-34-iscsi devstack@keystone.service[80254]: clock source: unix Jul 15 19:13:39.130056 dv-cinder-799871-34-iscsi devstack@keystone.service[80254]: pcre jit disabled Jul 15 19:13:39.130056 dv-cinder-799871-34-iscsi devstack@keystone.service[80254]: detected number of CPU cores: 4 Jul 15 19:13:39.130056 dv-cinder-799871-34-iscsi devstack@keystone.service[80254]: current working directory: / Jul 15 19:13:39.130056 dv-cinder-799871-34-iscsi devstack@keystone.service[80254]: detected binary path: /usr/bin/uwsgi-core Jul 15 19:13:39.130056 dv-cinder-799871-34-iscsi devstack@keystone.service[80254]: your processes number limit is 47680 Jul 15 19:13:39.130689 dv-cinder-799871-34-iscsi devstack@keystone.service[80254]: your memory page size is 4096 bytes Jul 15 19:13:39.130689 dv-cinder-799871-34-iscsi devstack@keystone.service[80254]: detected max file descriptor number: 2048 Jul 15 19:13:39.130689 dv-cinder-799871-34-iscsi devstack@keystone.service[80254]: lock engine: pthread robust mutexes Jul 15 19:13:39.130689 dv-cinder-799871-34-iscsi devstack@keystone.service[80254]: thunder lock: enabled Jul 15 19:13:39.130689 dv-cinder-799871-34-iscsi devstack@keystone.service[80254]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 15 19:13:39.130689 dv-cinder-799871-34-iscsi devstack@keystone.service[80254]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 15 19:13:39.171050 dv-cinder-799871-34-iscsi devstack@keystone.service[80254]: Python main interpreter initialized at 0x55608ec709c0 Jul 15 19:13:39.171050 dv-cinder-799871-34-iscsi devstack@keystone.service[80254]: python threads support enabled Jul 15 19:13:39.171050 dv-cinder-799871-34-iscsi devstack@keystone.service[80254]: your server socket listen backlog is limited to 100 connections Jul 15 19:13:39.171050 dv-cinder-799871-34-iscsi devstack@keystone.service[80254]: your mercy for graceful operations on workers is 90 seconds Jul 15 19:13:39.171437 dv-cinder-799871-34-iscsi devstack@keystone.service[80254]: mapped 403077 bytes (393 KB) for 2 cores Jul 15 19:13:39.171437 dv-cinder-799871-34-iscsi devstack@keystone.service[80254]: *** Operational MODE: preforking *** Jul 15 19:13:39.171819 dv-cinder-799871-34-iscsi devstack@keystone.service[80254]: *** uWSGI is running in multiple interpreter mode *** Jul 15 19:13:39.171819 dv-cinder-799871-34-iscsi devstack@keystone.service[80254]: spawned uWSGI master process (pid: 80254) Jul 15 19:13:39.171950 dv-cinder-799871-34-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 15 19:13:39.172633 dv-cinder-799871-34-iscsi devstack@keystone.service[80254]: spawned uWSGI worker 1 (pid: 80268, cores: 1) Jul 15 19:13:39.173947 dv-cinder-799871-34-iscsi devstack@keystone.service[80254]: spawned uWSGI worker 2 (pid: 80269, cores: 1) Jul 15 19:13:39.173947 dv-cinder-799871-34-iscsi devstack@keystone.service[80254]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 15 19:13:44.635804 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 15 19:13:44.635804 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: from cryptography.utils import int_from_bytes Jul 15 19:13:44.636574 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 15 19:13:44.636574 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: from cryptography.utils import int_from_bytes Jul 15 19:13:44.639303 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 15 19:13:44.639303 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: from cryptography.utils import int_from_bytes Jul 15 19:13:44.639763 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 15 19:13:44.639763 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: from cryptography.utils import int_from_bytes Jul 15 19:13:44.695012 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 15 19:13:44.695206 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 15 19:13:44.695384 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] command line args: [] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 15 19:13:44.695642 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 15 19:13:44.695727 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 15 19:13:44.695809 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ================================================================================ {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 15 19:13:44.695972 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 15 19:13:44.696067 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] admin_token = **** {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.696142 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] command line args: [] {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 15 19:13:44.696285 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] config_dir = [] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.696408 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 15 19:13:44.696495 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.696575 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ================================================================================ {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 15 19:13:44.696654 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] config_source = [] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.696808 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] debug = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.696892 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] admin_token = **** {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.697009 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.697288 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] config_dir = [] {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.697288 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.697413 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.697474 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] config_source = [] {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.697538 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.697605 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] debug = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.697812 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.697929 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.698052 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.698173 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.698292 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.698395 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] list_limit = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.698551 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.698675 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] log_config_append = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.698795 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.699103 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] list_limit = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.699283 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.699480 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] log_config_append = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.699632 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] log_dir = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.699728 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.699728 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] log_dir = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.699934 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] log_file = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.700015 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] log_file = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.700092 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.700211 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.700211 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.700397 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.700482 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.700599 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.700685 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.700761 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.700885 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.700963 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.701091 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.701174 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.701296 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.701411 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.701499 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.701650 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.701808 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.701899 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.701973 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.702134 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.702222 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.702371 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.702454 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.702624 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.702709 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.702869 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.702948 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.703116 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.703242 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.703395 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.703630 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.33/identity {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.703815 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] publish_errors = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.703940 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.33/identity {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.704049 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.704128 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] publish_errors = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.704278 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.704361 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.704458 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.704541 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.704672 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.704755 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.704917 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.705006 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.705149 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.705228 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] standard_threads = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.705387 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] standard_threads = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.705472 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.705617 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.705703 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.705825 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.705905 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.705992 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] use_journal = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.706067 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.706175 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] use_json = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.706255 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] use_journal = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.706355 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] use_stderr = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.706439 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] use_json = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.706587 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] use_stderr = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.706666 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] use_syslog = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.706738 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] use_syslog = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.706814 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.706912 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:13:44.706992 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.707167 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.707246 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.707404 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.707488 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.707663 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.707742 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.707901 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.707980 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.708149 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.708227 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.708384 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.708462 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.708630 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.708763 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.708867 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.708946 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.709113 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.709191 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.709357 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.709437 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.709602 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.709682 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.709859 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.709942 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.710104 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.710190 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.710338 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.710427 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.710583 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.710667 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.710827 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.710907 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.711064 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.711142 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.711316 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.711406 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.711566 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.711650 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.711809 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.711888 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.712045 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.712199 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.712282 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.712361 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.712543 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.712622 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.712791 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.712878 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.713037 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.713115 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.713266 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.713344 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.713494 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.713580 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.713722 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.713841 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.713998 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.714087 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.714244 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.714331 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.714489 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.714573 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.714718 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.714796 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.714947 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.715030 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.715196 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.715269 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.715436 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.715512 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.715691 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.715770 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.715931 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.715999 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.716180 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.716302 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.716412 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.716501 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.716648 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.716723 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.716885 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.716964 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.717120 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.717210 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.717356 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.717442 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.717596 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.717672 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.717893 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.717996 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.718119 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.718266 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.718356 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.718428 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.718585 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.718668 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.718826 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.718909 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.719067 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.719148 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.719320 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.719411 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.719555 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.719632 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.719798 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.719877 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.720041 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.720209 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.720336 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.720433 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.720551 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.720629 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.720732 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.720812 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.720966 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.721047 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.721218 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.721299 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.721457 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.721540 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.721699 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.721823 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.721962 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.722059 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.722247 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.722337 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.722462 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.722551 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.722698 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.722778 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.722939 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.723020 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.723180 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.723261 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.723451 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.723532 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.723694 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.723774 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.723950 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.724032 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.724198 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.724284 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.724437 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.724516 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.724676 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.724757 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.724914 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.724993 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.725162 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.725241 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.725406 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.725499 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.725645 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.725724 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.725917 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.725998 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.726158 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.726250 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.726398 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.726478 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.726636 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.726715 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.726875 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.726962 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.727113 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.727200 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.727368 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.727453 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.727608 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.727695 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.727853 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.727938 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.728094 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.728173 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.728336 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.728567 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.728649 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.728884 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.729393 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.730055 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.730132 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.730220 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.730297 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.730377 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.730451 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.730524 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.730596 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.730685 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.730760 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.730832 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.730961 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.731042 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.731203 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.731292 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.731455 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.731535 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.731697 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.731778 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.731959 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.732047 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.732310 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.732389 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.732552 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.732632 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.732807 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] auth.external = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.732895 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.733048 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.733186 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.733353 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.733434 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.733593 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.733681 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] auth.external = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.733903 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.733998 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] auth.password = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.734160 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] auth.token = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.734240 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.734413 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.734498 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.734647 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] auth.password = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.734736 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.734882 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] auth.token = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.734964 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.735123 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.735204 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.735359 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.735448 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.735612 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.735693 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.735845 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.735925 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.736083 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] credential.caching = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.736174 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.736325 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.736405 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.736556 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.736637 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.736790 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] credential.caching = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.736880 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.737035 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.737160 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.737281 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.737361 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.737522 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.737613 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.737756 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.737870 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.738017 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.738099 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.738252 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.738347 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.738516 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.738599 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.738767 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.738848 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.739003 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.739162 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.739240 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.739313 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.739521 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.739603 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.739766 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.739856 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.740004 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] federation.caching = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.740084 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.740247 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.740328 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.740498 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.740580 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.740729 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] federation.caching = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.740876 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.740956 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.741029 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.741197 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.741280 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.741436 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.741516 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.741664 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.741743 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.741933 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.742016 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.742171 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.742253 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.742420 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.742499 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.742653 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.742740 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.742895 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] identity.caching = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.742981 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.743128 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.743208 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.743374 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.743457 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.743610 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.743690 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] identity.caching = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.743849 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.743930 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.744092 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.744173 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.744327 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.744409 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.744562 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.744642 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.744806 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.744888 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.745055 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.745196 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.745837 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.745922 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.746074 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.746156 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.746320 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.746401 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.746549 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.746629 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.746787 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.746870 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.747018 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.747100 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.747262 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.747343 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.747508 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.747590 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.747752 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.747833 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.748004 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.748085 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.748246 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.748328 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.748484 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.748575 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.748725 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.748806 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.748958 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.749040 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.749185 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.749254 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.749420 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.749500 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.749648 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.749736 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.749904 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.749988 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.750143 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.750238 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.750381 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.750461 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.750626 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.750708 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.750855 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.750944 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.751102 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.751182 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.751345 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.751424 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.751576 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.751664 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.751816 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.751896 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.752047 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.752126 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.752276 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.752364 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.752515 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.752595 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.752741 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.752820 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.752975 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.753065 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.753210 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.753296 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.753447 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.753532 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.753684 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.753791 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.753941 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.754022 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.754173 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.754252 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.754421 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.754511 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.754650 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.754731 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.754877 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.754958 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.755119 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.755207 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.755358 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.755447 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.755590 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.755670 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.user = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.755831 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.755921 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.756065 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.756147 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.756294 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.756375 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.756532 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.756622 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.756784 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.756864 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.757011 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.757091 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.757256 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.757344 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.757487 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.757567 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.757724 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.757884 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.757982 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.758074 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.758212 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.758299 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.758445 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.758525 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.758675 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.758763 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.758909 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.758990 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.759138 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.759219 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.759385 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.759473 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.759628 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.759710 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.759859 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.759938 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.760108 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.760197 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.760347 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.760428 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.760586 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.760740 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.760869 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.761004 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.761132 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.761258 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.761383 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.761508 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.761632 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.761787 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.761787 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.762029 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.762159 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.762287 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.762427 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.762637 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.762765 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.762892 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.763020 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.763148 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.763277 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.763411 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.763539 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.763667 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.763794 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.763921 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.764052 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.764187 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.764328 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.764455 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.764582 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.764710 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.764836 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.764971 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] resource.caching = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.765098 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.765227 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.765355 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.765482 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.765609 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] resource.caching = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.765750 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.765926 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.766065 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.766213 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.766352 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.766499 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.766638 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.766638 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.766638 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.766638 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.767099 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] role.caching = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.767099 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] role.driver = sql {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.767463 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.767908 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.768031 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.768155 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.768280 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.768421 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.768547 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.768678 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.768803 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.768927 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.769049 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.769182 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.769306 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.769430 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] role.caching = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.769554 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.769678 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] role.driver = sql {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.769850 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.769987 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.770126 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.770254 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.770381 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.770508 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.770637 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.770773 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.770900 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.771028 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.771156 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.771283 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.771410 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.771545 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.771674 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.771812 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.771939 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.772067 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.772203 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.772341 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.772473 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.772602 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.772732 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.772862 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.772991 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.773128 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.773257 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.773386 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.773524 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.773654 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.773812 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.773951 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.774081 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.774210 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.774338 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.774476 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.774614 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.774746 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.774875 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.775004 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.775132 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.775261 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.775407 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.775539 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.775922 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.776051 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.776181 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.776822 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.776968 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.777099 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.777228 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] token.caching = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.777356 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.777484 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.777613 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.777749 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.777912 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.778040 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.778169 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.778297 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.778425 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.778562 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.778688 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.778816 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.778945 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.779074 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.779201 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.779336 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.779464 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] token.caching = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.779591 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.779728 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.779856 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.779984 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.780121 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.780249 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.780376 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.780503 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.780631 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.780758 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.780893 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.781021 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.781149 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.781277 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.781413 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.781541 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.781676 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.781816 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.781939 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] database.connection = **** {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.782079 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.782216 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.782342 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.782469 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.782604 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.782732 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.782859 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.782986 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.783114 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.783252 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.783386 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.783512 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.783640 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.783768 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.783895 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.784022 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.784157 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.connection = **** {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.784296 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.784507 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.784635 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.784763 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.784889 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.785016 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.785151 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.785288 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.785417 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.785542 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.785667 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.785843 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.785967 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.786106 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.786233 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.786362 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.786489 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.786614 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.786741 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.786876 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.787014 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 15 19:13:44.787142 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.787142 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.787142 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.787142 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.787142 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.787142 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.787142 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.787142 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:13:44.787142 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 15 19:13:44.816418 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:13:44.816655 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:13:44.816803 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:13:44.816934 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:13:44.817068 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:13:44.817255 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:13:44.825409 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:13:44.825697 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:13:44.854015 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:13:44.854200 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:13:44.854346 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:13:44.854510 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:13:44.996621 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:13:44.996888 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:13:45.046014 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:13:45.046260 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:13:45.046597 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:13:45.046833 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:13:45.111049 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:13:45.111276 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:13:45.111437 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:13:45.111582 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:13:45.111734 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:13:45.111870 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:13:45.111999 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:13:45.112133 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:13:45.112274 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:13:45.112408 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:13:45.112539 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:13:45.112670 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:13:45.137133 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:13:45.137365 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:13:45.137498 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:13:45.137639 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:13:45.137800 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:13:45.137885 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:13:45.137970 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:13:45.138112 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:13:45.138322 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:13:45.138454 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:13:45.138540 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:13:45.138664 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:13:45.138800 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:13:45.138800 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:13:45.149463 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:13:45.149988 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:13:45.162770 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:13:45.163000 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:13:45.163224 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:13:45.163352 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:13:45.163481 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:13:45.163607 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:13:45.163745 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:13:45.163875 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:13:45.164014 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:13:45.164151 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:13:45.164279 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:13:45.164415 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:13:45.164415 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:13:45.164653 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:13:45.180985 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.181190 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 15 19:13:45.181321 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.181521 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.181653 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 15 19:13:45.181832 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 15 19:13:45.181918 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.182075 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 15 19:13:45.182345 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.182608 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.182696 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.182874 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.182958 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.183125 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.183217 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.183378 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.193944 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.194217 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.194311 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.194477 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.194563 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.194748 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.194832 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.195098 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.203639 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 19:13:45.204089 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80268) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 19:13:45.206934 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 19:13:45.207212 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.207437 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80268) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 19:13:45.207521 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.207724 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.207810 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.208008 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.208092 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.208260 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.208359 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.208524 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.208610 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.208791 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.208877 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.209055 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.209148 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.209321 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.209412 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.209585 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.209667 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.209931 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.210136 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.236786 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.237390 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.238428 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 19:13:45.238705 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.239071 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80268) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 19:13:45.239563 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.244035 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 19:13:45.244315 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.244591 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.244684 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80268) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 19:13:45.244929 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.245238 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.252162 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 19:13:45.252398 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.252739 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80268) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 19:13:45.253022 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.256868 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 19:13:45.257153 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.257430 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.257532 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80268) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 19:13:45.257709 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.257829 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.258002 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.258095 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.258274 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.258361 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.258542 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.258625 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.258806 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.258889 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.259155 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.259427 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.278996 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.279269 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.279669 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.279949 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.283935 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 19:13:45.284672 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80268) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 19:13:45.287612 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.287903 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.288480 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.288778 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.293070 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 19:13:45.294016 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80268) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 19:13:45.296783 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 19:13:45.297735 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80268) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 19:13:45.300431 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.300726 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.301005 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.301293 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.301424 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.301566 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.301697 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.301877 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.302078 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.302368 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.302666 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.302956 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.320942 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 19:13:45.321221 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.321484 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.321755 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.322177 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80268) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 19:13:45.322461 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.322950 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.323081 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.330392 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.331527 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.332081 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.332353 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.333257 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.333534 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.335547 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 19:13:45.335835 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.336122 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.336400 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.336678 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.336763 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80268) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 19:13:45.337057 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.337352 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.337641 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.337944 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.350522 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 19:13:45.350798 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.351072 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.351345 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.351620 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.351902 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.351986 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80268) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 19:13:45.352173 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.352257 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.352510 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.352791 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.353071 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.353356 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.353632 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.371403 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 19:13:45.373100 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80268) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 19:13:45.374691 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 19:13:45.376410 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80268) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 19:13:45.378009 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.379610 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.379775 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.381391 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 19:13:45.381476 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.381667 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.381966 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.383206 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80268) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 19:13:45.383487 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.383767 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.389753 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.391381 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 19:13:45.391648 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.393361 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80268) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 19:13:45.394714 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.394999 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.395282 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.395552 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.396638 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.396929 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.397221 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.397503 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.405593 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 19:13:45.405902 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.406178 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.407734 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80268) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 19:13:45.408023 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.408311 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.414043 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 19:13:45.414319 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.414593 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.414864 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.415140 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.415421 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.415692 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.415958 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.416219 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80268) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 19:13:45.416393 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.416486 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.416667 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.416752 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.416969 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.417061 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.417241 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.417333 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.417506 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.417591 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.417795 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.417883 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.418163 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.418541 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.418826 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.419111 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.419386 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.419656 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.419933 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:13:45.456332 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55608ec709c0 pid: 80269 (default app) Jul 15 19:13:45.458437 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55608ec709c0 pid: 80268 (default app) Jul 15 19:13:45.459729 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1d2941-132f-43e6-addf-a1d30af909c9 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:45.460017 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1d2941-132f-43e6-addf-a1d30af909c9 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:45.460308 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1d2941-132f-43e6-addf-a1d30af909c9 None None] PATH_INFO: `/v3` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:45.465196 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1/1] 10.222.0.33 () {50 vars in 791 bytes} [Fri Jul 15 19:13:45 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:13:54.057107 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937bf91e-799f-4e2e-bece-b57b4bfb8130 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:54.057643 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937bf91e-799f-4e2e-bece-b57b4bfb8130 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:54.057643 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937bf91e-799f-4e2e-bece-b57b4bfb8130 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:54.061916 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1/2] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:13:54 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 19:13:54.067094 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31bb8c0-9e9b-47ef-a47f-a77aaf235ec5 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:54.067478 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31bb8c0-9e9b-47ef-a47f-a77aaf235ec5 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:54.067639 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31bb8c0-9e9b-47ef-a47f-a77aaf235ec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:54.134810 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: INFO dbcounter [None req-f31bb8c0-9e9b-47ef-a47f-a77aaf235ec5 None None] Registered counter for database keystone Jul 15 19:13:54.156749 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG oslo_db.sqlalchemy.engines [None req-f31bb8c0-9e9b-47ef-a47f-a77aaf235ec5 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80269) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 15 19:13:54.162930 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writer thread running {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 15 19:13:54.336252 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG passlib.handlers.bcrypt [None req-f31bb8c0-9e9b-47ef-a47f-a77aaf235ec5 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80269) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 15 19:13:54.336589 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG passlib.handlers.bcrypt [None req-f31bb8c0-9e9b-47ef-a47f-a77aaf235ec5 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80269) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 15 19:13:54.390892 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-f31bb8c0-9e9b-47ef-a47f-a77aaf235ec5 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:13:54.392505 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-f31bb8c0-9e9b-47ef-a47f-a77aaf235ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:13:54.514097 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 2/3] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:13:54 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 448 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 15 19:13:54.667807 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937bf91e-799f-4e2e-bece-b57b4bfb8130 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:54.740777 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: INFO dbcounter [None req-937bf91e-799f-4e2e-bece-b57b4bfb8130 None None] Registered counter for database keystone Jul 15 19:13:54.761877 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG oslo_db.sqlalchemy.engines [None req-937bf91e-799f-4e2e-bece-b57b4bfb8130 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80268) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 15 19:13:54.767997 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writer thread running {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 15 19:13:54.898197 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937bf91e-799f-4e2e-bece-b57b4bfb8130 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:54.902796 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c4f97c-4aed-4772-b087-8e0703b9a78f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:54.903458 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c4f97c-4aed-4772-b087-8e0703b9a78f admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:54.903744 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c4f97c-4aed-4772-b087-8e0703b9a78f admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:54.904022 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c4f97c-4aed-4772-b087-8e0703b9a78f admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:54.912249 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c4f97c-4aed-4772-b087-8e0703b9a78f admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:54.965620 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: WARNING keystone.common.rbac_enforcer.enforcer [None req-99c4f97c-4aed-4772-b087-8e0703b9a78f admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 15 19:13:54.965949 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c4f97c-4aed-4772-b087-8e0703b9a78f admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:54.975600 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 2/4] 10.222.0.33 () {58 vars in 1324 bytes} [Fri Jul 15 19:13:54 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 19:13:54.982287 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31bb8c0-9e9b-47ef-a47f-a77aaf235ec5 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:54.997190 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31bb8c0-9e9b-47ef-a47f-a77aaf235ec5 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:55.001602 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ecb9aa-26d1-4364-bd4d-6ff25670518b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:55.002751 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ecb9aa-26d1-4364-bd4d-6ff25670518b admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:55.003041 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ecb9aa-26d1-4364-bd4d-6ff25670518b admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:55.003322 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ecb9aa-26d1-4364-bd4d-6ff25670518b admin admin] PATH_INFO: `/v3/projects/392b3b5584be451c8c287eb751b4d7de` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:55.007426 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ecb9aa-26d1-4364-bd4d-6ff25670518b admin admin] RBAC: Authorizing `identity:get_project(project_id=392b3b5584be451c8c287eb751b4d7de)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:55.059589 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: WARNING keystone.common.rbac_enforcer.enforcer [None req-42ecb9aa-26d1-4364-bd4d-6ff25670518b admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 15 19:13:55.059932 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ecb9aa-26d1-4364-bd4d-6ff25670518b admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:55.064514 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 3/5] 10.222.0.33 () {56 vars in 1207 bytes} [Fri Jul 15 19:13:54 2022] GET /identity/v3/projects/392b3b5584be451c8c287eb751b4d7de => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:13:57.057813 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf41eb14-dec7-4c7a-984b-af9f14e2c564 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:57.058319 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf41eb14-dec7-4c7a-984b-af9f14e2c564 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:57.058319 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf41eb14-dec7-4c7a-984b-af9f14e2c564 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:57.062405 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 3/6] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:13:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:13:57.067794 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d9467d-b13c-4f46-835a-e61a299ff0bc None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:57.068217 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d9467d-b13c-4f46-835a-e61a299ff0bc None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:57.068357 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d9467d-b13c-4f46-835a-e61a299ff0bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:57.086849 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-f0d9467d-b13c-4f46-835a-e61a299ff0bc None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:13:57.088904 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-f0d9467d-b13c-4f46-835a-e61a299ff0bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:13:57.096796 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 4/7] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:13:57 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 15 19:13:57.118743 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf41eb14-dec7-4c7a-984b-af9f14e2c564 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:57.130627 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf41eb14-dec7-4c7a-984b-af9f14e2c564 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:57.134763 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202931f2-d7fe-4dc2-99a5-36f6a75a97ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:57.135893 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202931f2-d7fe-4dc2-99a5-36f6a75a97ef admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:57.136174 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202931f2-d7fe-4dc2-99a5-36f6a75a97ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:57.136449 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202931f2-d7fe-4dc2-99a5-36f6a75a97ef admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:57.144422 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202931f2-d7fe-4dc2-99a5-36f6a75a97ef admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:57.146232 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202931f2-d7fe-4dc2-99a5-36f6a75a97ef admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:57.155329 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 4/8] 10.222.0.33 () {58 vars in 1324 bytes} [Fri Jul 15 19:13:57 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 19:13:57.160591 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d9467d-b13c-4f46-835a-e61a299ff0bc None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:57.170074 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d9467d-b13c-4f46-835a-e61a299ff0bc None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:57.174467 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabc956b-7529-4e25-9c94-9899d6536300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:57.175325 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabc956b-7529-4e25-9c94-9899d6536300 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:57.175612 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabc956b-7529-4e25-9c94-9899d6536300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:57.175891 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabc956b-7529-4e25-9c94-9899d6536300 admin admin] PATH_INFO: `/v3/users/96f47b71ea5a45ad92368f6b4130c36c` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:57.177279 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabc956b-7529-4e25-9c94-9899d6536300 admin admin] RBAC: Authorizing `identity:get_user(user_id=96f47b71ea5a45ad92368f6b4130c36c)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:57.179127 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabc956b-7529-4e25-9c94-9899d6536300 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:57.180701 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 5/9] 10.222.0.33 () {56 vars in 1198 bytes} [Fri Jul 15 19:13:57 2022] GET /identity/v3/users/96f47b71ea5a45ad92368f6b4130c36c => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:13:59.266919 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc46d19-886f-4e72-b4fc-8706d726db37 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:59.267420 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc46d19-886f-4e72-b4fc-8706d726db37 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:59.267420 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc46d19-886f-4e72-b4fc-8706d726db37 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:59.268832 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 5/10] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:13:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:13:59.269798 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64fcb36-8b91-4bdb-8d0a-1ca4c992bf3d None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:59.270208 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64fcb36-8b91-4bdb-8d0a-1ca4c992bf3d None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:59.270366 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64fcb36-8b91-4bdb-8d0a-1ca4c992bf3d None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:59.271715 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 6/11] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:13:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 19:13:59.273870 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff8b474-59bc-4f85-8ed1-3e451680b784 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:59.274153 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff8b474-59bc-4f85-8ed1-3e451680b784 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:59.274424 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff8b474-59bc-4f85-8ed1-3e451680b784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:59.277557 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb167f-3cfa-4ce6-b752-7dca75850ac2 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:59.277882 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb167f-3cfa-4ce6-b752-7dca75850ac2 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:59.278169 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb167f-3cfa-4ce6-b752-7dca75850ac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:59.295227 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-3dbb167f-3cfa-4ce6-b752-7dca75850ac2 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:13:59.296431 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-3dbb167f-3cfa-4ce6-b752-7dca75850ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:13:59.315683 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG passlib.handlers.bcrypt [None req-fff8b474-59bc-4f85-8ed1-3e451680b784 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80268) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 15 19:13:59.316211 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG passlib.handlers.bcrypt [None req-fff8b474-59bc-4f85-8ed1-3e451680b784 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80268) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 15 19:13:59.362038 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-fff8b474-59bc-4f85-8ed1-3e451680b784 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:13:59.363619 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-fff8b474-59bc-4f85-8ed1-3e451680b784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:13:59.377230 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 7/12] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:13:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 19:13:59.400033 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbb167f-3cfa-4ce6-b752-7dca75850ac2 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:59.423425 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:59.463825 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 6/13] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:13:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:13:59.486012 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff8b474-59bc-4f85-8ed1-3e451680b784 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:59.510246 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbb167f-3cfa-4ce6-b752-7dca75850ac2 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:59.544109 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728ffd54-4fb9-4f81-8a82-2e9463676562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:59.544836 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728ffd54-4fb9-4f81-8a82-2e9463676562 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:59.545119 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728ffd54-4fb9-4f81-8a82-2e9463676562 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:59.545391 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728ffd54-4fb9-4f81-8a82-2e9463676562 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:59.556333 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728ffd54-4fb9-4f81-8a82-2e9463676562 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:13:59.567935 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-728ffd54-4fb9-4f81-8a82-2e9463676562 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:13:59.567935 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:13:59.567935 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:13:59.567935 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:13:59.567935 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:13:59.567935 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:13:59.567935 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:13:59.567935 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:13:59.567935 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:13:59.567935 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:13:59.567935 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:13:59.567935 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:13:59.567935 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:13:59.567935 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:13:59.567935 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:13:59.567935 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:13:59.568950 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:13:59.568950 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:13:59.568950 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:13:59.568950 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:13:59.568950 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:13:59.568950 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:13:59.568950 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:13:59.568950 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:13:59.568950 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:13:59.568950 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:13:59.568950 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:13:59.568950 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:13:59.568950 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:13:59.568950 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:13:59.568950 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:13:59.568950 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:13:59.569877 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:13:59.569877 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:13:59.569877 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:13:59.569877 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:13:59.569877 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:13:59.569877 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:13:59.569877 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:13:59.569877 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:13:59.569877 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:13:59.569877 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:13:59.569877 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:13:59.569877 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:13:59.573911 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 8/14] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jul 15 19:13:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:13:59.575112 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728ffd54-4fb9-4f81-8a82-2e9463676562 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:59.586929 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff8b474-59bc-4f85-8ed1-3e451680b784 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:59.589655 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:59.625674 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3f0d66-6161-4a48-8479-2c1ad0994766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:59.626595 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3f0d66-6161-4a48-8479-2c1ad0994766 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:59.626881 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3f0d66-6161-4a48-8479-2c1ad0994766 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:59.627158 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3f0d66-6161-4a48-8479-2c1ad0994766 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:59.657533 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:59.675517 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728ffd54-4fb9-4f81-8a82-2e9463676562 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:59.697047 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3f0d66-6161-4a48-8479-2c1ad0994766 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:59.698602 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3f0d66-6161-4a48-8479-2c1ad0994766 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:59.710670 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913635ca-d7cd-442a-b0ae-8cb5a2d8142f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:59.711340 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913635ca-d7cd-442a-b0ae-8cb5a2d8142f None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:59.711622 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913635ca-d7cd-442a-b0ae-8cb5a2d8142f None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:59.711909 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913635ca-d7cd-442a-b0ae-8cb5a2d8142f None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:59.712677 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913635ca-d7cd-442a-b0ae-8cb5a2d8142f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:13:59.712985 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913635ca-d7cd-442a-b0ae-8cb5a2d8142f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:59.714410 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913635ca-d7cd-442a-b0ae-8cb5a2d8142f None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:59.726363 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 9/15] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 15 19:13:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:13:59.732326 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913635ca-d7cd-442a-b0ae-8cb5a2d8142f None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:59.799686 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 7/16] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 15 19:13:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 15 19:13:59.805174 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3f0d66-6161-4a48-8479-2c1ad0994766 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:59.808151 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:59.831800 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913635ca-d7cd-442a-b0ae-8cb5a2d8142f None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:59.867200 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65696855-74e7-4d36-8c87-a398c2566c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:59.868079 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65696855-74e7-4d36-8c87-a398c2566c88 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:59.868363 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65696855-74e7-4d36-8c87-a398c2566c88 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:59.868645 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65696855-74e7-4d36-8c87-a398c2566c88 None admin] PATH_INFO: `/v3/users/96f47b71ea5a45ad92368f6b4130c36c` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:59.869981 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65696855-74e7-4d36-8c87-a398c2566c88 None admin] RBAC: Authorizing `identity:get_user(user_id=96f47b71ea5a45ad92368f6b4130c36c)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:59.871449 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65696855-74e7-4d36-8c87-a398c2566c88 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:59.873088 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 10/17] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 15 19:13:59 2022] GET /identity/v3/users/96f47b71ea5a45ad92368f6b4130c36c => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:13:59.878434 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65696855-74e7-4d36-8c87-a398c2566c88 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:59.893708 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:59.903223 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3f0d66-6161-4a48-8479-2c1ad0994766 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:59.937338 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5caf98-3aee-414d-8a96-ef4fba223485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:59.938172 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5caf98-3aee-414d-8a96-ef4fba223485 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:59.938456 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5caf98-3aee-414d-8a96-ef4fba223485 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:59.938737 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5caf98-3aee-414d-8a96-ef4fba223485 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:59.958743 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5caf98-3aee-414d-8a96-ef4fba223485 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:59.960202 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5caf98-3aee-414d-8a96-ef4fba223485 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:59.979804 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65696855-74e7-4d36-8c87-a398c2566c88 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:59.981402 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-ac5caf98-3aee-414d-8a96-ef4fba223485 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:13:59.981402 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:13:59.981402 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:13:59.981402 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:13:59.981402 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:13:59.981402 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:13:59.981402 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:13:59.981402 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:13:59.981402 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:13:59.981402 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:13:59.981402 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:13:59.981402 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:13:59.981402 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:13:59.981402 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:13:59.981402 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:13:59.982308 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:13:59.982308 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:13:59.982308 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:13:59.982308 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:13:59.982308 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:13:59.982308 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:13:59.982308 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:13:59.982308 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:13:59.982308 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:13:59.982308 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:13:59.982308 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:13:59.982308 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:13:59.982308 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:13:59.982308 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:13:59.982308 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:13:59.982308 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:13:59.982308 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:13:59.983320 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:13:59.983320 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:13:59.983320 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:13:59.983320 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:13:59.983320 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 8/18] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:13:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:13:59.988355 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5caf98-3aee-414d-8a96-ef4fba223485 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:00.017011 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106728c9-621d-4dae-b8b2-601dfcb300a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:00.017831 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106728c9-621d-4dae-b8b2-601dfcb300a2 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:00.018115 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106728c9-621d-4dae-b8b2-601dfcb300a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:00.018387 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106728c9-621d-4dae-b8b2-601dfcb300a2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:00.019580 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106728c9-621d-4dae-b8b2-601dfcb300a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:00.021022 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106728c9-621d-4dae-b8b2-601dfcb300a2 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:00.022654 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 11/19] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:13:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:14:00.029474 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106728c9-621d-4dae-b8b2-601dfcb300a2 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:00.045373 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:00.087857 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5caf98-3aee-414d-8a96-ef4fba223485 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:00.110905 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:00.123383 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc004eb-70d7-4119-a02c-e0fce9c0e567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:00.124042 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc004eb-70d7-4119-a02c-e0fce9c0e567 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:00.124323 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc004eb-70d7-4119-a02c-e0fce9c0e567 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:00.124599 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc004eb-70d7-4119-a02c-e0fce9c0e567 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:00.125321 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc004eb-70d7-4119-a02c-e0fce9c0e567 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:00.125626 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc004eb-70d7-4119-a02c-e0fce9c0e567 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:00.127038 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc004eb-70d7-4119-a02c-e0fce9c0e567 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:00.129947 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106728c9-621d-4dae-b8b2-601dfcb300a2 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:00.144495 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 9/20] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:13:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:14:00.166221 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee90e97-a84d-4082-be8d-6e96fe7826b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:00.166906 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee90e97-a84d-4082-be8d-6e96fe7826b4 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:00.167187 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee90e97-a84d-4082-be8d-6e96fe7826b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:00.167464 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee90e97-a84d-4082-be8d-6e96fe7826b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:00.168330 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee90e97-a84d-4082-be8d-6e96fe7826b4 None admin] RBAC: Adding query filter params (role.id=4318f86ba9f74cd4a0bef1c311276657, scope.domain.id=default, user.id=96f47b71ea5a45ad92368f6b4130c36c) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:00.168644 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee90e97-a84d-4082-be8d-6e96fe7826b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:00.170248 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee90e97-a84d-4082-be8d-6e96fe7826b4 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:00.177992 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 12/21] 10.222.0.33 () {56 vars in 1322 bytes} [Fri Jul 15 19:14:00 2022] GET /identity/v3/role_assignments?user.id=96f47b71ea5a45ad92368f6b4130c36c&scope.domain.id=default&role.id=4318f86ba9f74cd4a0bef1c311276657 => generated 233 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:14:02.502792 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5955c16d-00c5-4892-8d2e-dfe33d7a98f9 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:02.503337 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5955c16d-00c5-4892-8d2e-dfe33d7a98f9 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:02.503337 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5955c16d-00c5-4892-8d2e-dfe33d7a98f9 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:02.504731 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 10/22] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:14:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:14:02.510138 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d14a3-d238-4759-9f5e-ac7348834875 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:02.510550 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d14a3-d238-4759-9f5e-ac7348834875 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:02.510746 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d14a3-d238-4759-9f5e-ac7348834875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:02.538781 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-072d14a3-d238-4759-9f5e-ac7348834875 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:14:02.540017 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-072d14a3-d238-4759-9f5e-ac7348834875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:14:02.627134 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 13/23] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:14:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:14:02.670114 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5955c16d-00c5-4892-8d2e-dfe33d7a98f9 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:02.797084 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f4b663-320b-4bba-aea6-441c3162698f None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:02.797267 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f4b663-320b-4bba-aea6-441c3162698f None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:02.797556 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f4b663-320b-4bba-aea6-441c3162698f None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:02.798963 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 14/24] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:14:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:14:02.813801 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba80008-ee04-4d55-ab60-ead8f3c85c3e None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:02.813801 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba80008-ee04-4d55-ab60-ead8f3c85c3e None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:02.813801 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba80008-ee04-4d55-ab60-ead8f3c85c3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:02.833067 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-2ba80008-ee04-4d55-ab60-ead8f3c85c3e None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:14:02.834276 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-2ba80008-ee04-4d55-ab60-ead8f3c85c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:14:02.836760 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5955c16d-00c5-4892-8d2e-dfe33d7a98f9 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:02.865911 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:02.899229 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1effad4c-3b7b-4917-9d66-5a1b80d6986a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:02.900292 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1effad4c-3b7b-4917-9d66-5a1b80d6986a None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:02.901091 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1effad4c-3b7b-4917-9d66-5a1b80d6986a None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:02.901369 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1effad4c-3b7b-4917-9d66-5a1b80d6986a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:02.925610 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1effad4c-3b7b-4917-9d66-5a1b80d6986a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:14:02.934629 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-1effad4c-3b7b-4917-9d66-5a1b80d6986a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:14:02.934629 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:14:02.934629 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:14:02.934629 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:14:02.934629 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:14:02.934629 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:14:02.934629 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:14:02.934629 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:14:02.934629 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:14:02.934629 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:14:02.934629 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:14:02.934629 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:14:02.934629 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:14:02.934629 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:14:02.934629 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:14:02.934629 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:14:02.935470 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:14:02.935470 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:14:02.935470 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:14:02.935470 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:14:02.935470 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:14:02.935470 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:14:02.935470 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:14:02.935470 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:14:02.935470 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:14:02.935470 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:14:02.935470 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:14:02.935470 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:14:02.935470 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:14:02.935470 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:14:02.935470 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:14:02.935470 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:14:02.936235 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:14:02.936235 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:14:02.936235 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:14:02.936235 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:14:02.936235 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:14:02.936235 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:14:02.936235 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:14:02.936235 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:14:02.936235 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:14:02.936235 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:14:02.936235 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:14:02.936235 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:14:02.936235 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 11/25] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jul 15 19:14:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:14:02.950529 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1effad4c-3b7b-4917-9d66-5a1b80d6986a None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:02.984724 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 15/26] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:14:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 19:14:03.014593 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba80008-ee04-4d55-ab60-ead8f3c85c3e None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:03.017594 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:03.098522 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1effad4c-3b7b-4917-9d66-5a1b80d6986a None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:03.150232 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d99420f-979f-43dd-8e3e-d1ea79f7e477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:03.152042 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d99420f-979f-43dd-8e3e-d1ea79f7e477 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:03.152539 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d99420f-979f-43dd-8e3e-d1ea79f7e477 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:03.153038 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d99420f-979f-43dd-8e3e-d1ea79f7e477 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:03.154140 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d99420f-979f-43dd-8e3e-d1ea79f7e477 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:03.154662 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d99420f-979f-43dd-8e3e-d1ea79f7e477 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:03.156363 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d99420f-979f-43dd-8e3e-d1ea79f7e477 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:03.169931 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 12/27] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 15 19:14:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:03.171694 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9de5c1-1776-4006-957f-5cf4d4c392ff None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:03.172185 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9de5c1-1776-4006-957f-5cf4d4c392ff None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:03.172663 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9de5c1-1776-4006-957f-5cf4d4c392ff None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:03.174156 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 13/28] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:14:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:14:03.175452 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9de5c1-1776-4006-957f-5cf4d4c392ff None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:03.200243 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba80008-ee04-4d55-ab60-ead8f3c85c3e None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:03.251207 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:03.255283 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7761471-6dc2-4efd-9ac7-0b5fcfc17c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:03.256227 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7761471-6dc2-4efd-9ac7-0b5fcfc17c11 None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:03.256702 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7761471-6dc2-4efd-9ac7-0b5fcfc17c11 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:03.257152 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7761471-6dc2-4efd-9ac7-0b5fcfc17c11 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:03.258095 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7761471-6dc2-4efd-9ac7-0b5fcfc17c11 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:03.259650 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7761471-6dc2-4efd-9ac7-0b5fcfc17c11 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:03.319401 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9de5c1-1776-4006-957f-5cf4d4c392ff None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:03.354789 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1a61e8-2550-41e2-8491-858c1d79b20e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:03.355589 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1a61e8-2550-41e2-8491-858c1d79b20e None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:03.355868 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1a61e8-2550-41e2-8491-858c1d79b20e None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:03.356141 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1a61e8-2550-41e2-8491-858c1d79b20e None admin] PATH_INFO: `/v3/users/96f47b71ea5a45ad92368f6b4130c36c` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:03.357689 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1a61e8-2550-41e2-8491-858c1d79b20e None admin] RBAC: Authorizing `identity:get_user(user_id=96f47b71ea5a45ad92368f6b4130c36c)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:03.359202 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1a61e8-2550-41e2-8491-858c1d79b20e None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:03.365753 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 14/29] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 15 19:14:03 2022] GET /identity/v3/users/96f47b71ea5a45ad92368f6b4130c36c => generated 247 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:03.365753 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc616bc-8515-4b6d-8b96-a60a9509273c None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:03.365753 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc616bc-8515-4b6d-8b96-a60a9509273c None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:03.365753 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc616bc-8515-4b6d-8b96-a60a9509273c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:03.381493 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-6bc616bc-8515-4b6d-8b96-a60a9509273c None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:14:03.382758 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-6bc616bc-8515-4b6d-8b96-a60a9509273c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:14:03.461283 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 15/30] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:14:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:14:03.462967 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f5ae69-781d-49ad-a521-233eda5f2b79 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:03.463934 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f5ae69-781d-49ad-a521-233eda5f2b79 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:03.464354 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f5ae69-781d-49ad-a521-233eda5f2b79 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:03.465785 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 16/31] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:14:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:14:03.466732 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f5ae69-781d-49ad-a521-233eda5f2b79 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:03.511334 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:03.537050 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 16/32] 10.222.0.33 () {60 vars in 1131 bytes} [Fri Jul 15 19:14:03 2022] POST /identity/v3/roles => generated 219 bytes in 523 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:14:03.539443 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb5beb-ccaa-4b9d-88e3-140b3b70eaa9 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:03.539945 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb5beb-ccaa-4b9d-88e3-140b3b70eaa9 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:03.540427 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb5beb-ccaa-4b9d-88e3-140b3b70eaa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:03.559892 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-3cdb5beb-ccaa-4b9d-88e3-140b3b70eaa9 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:14:03.561855 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-3cdb5beb-ccaa-4b9d-88e3-140b3b70eaa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:14:03.595711 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f5ae69-781d-49ad-a521-233eda5f2b79 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:03.630355 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:03.640315 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 17/33] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:14:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:14:03.641635 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdb5beb-ccaa-4b9d-88e3-140b3b70eaa9 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:03.649677 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430ea83f-0bb1-4dd8-839e-7cde21b9e33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:03.651222 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ea83f-0bb1-4dd8-839e-7cde21b9e33a None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:03.651784 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ea83f-0bb1-4dd8-839e-7cde21b9e33a None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:03.652303 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ea83f-0bb1-4dd8-839e-7cde21b9e33a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:03.653892 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430ea83f-0bb1-4dd8-839e-7cde21b9e33a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:03.655786 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430ea83f-0bb1-4dd8-839e-7cde21b9e33a None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:03.657857 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 17/34] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:14:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:03.671634 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430ea83f-0bb1-4dd8-839e-7cde21b9e33a None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:03.827468 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdb5beb-ccaa-4b9d-88e3-140b3b70eaa9 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:03.832712 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430ea83f-0bb1-4dd8-839e-7cde21b9e33a None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:03.837721 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:03.871914 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25502dbf-f964-42ed-836a-4d090b0df825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:03.872763 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6c3bc0-5018-4056-b1fb-43aa1b233879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:03.872945 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25502dbf-f964-42ed-836a-4d090b0df825 None admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:03.873161 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25502dbf-f964-42ed-836a-4d090b0df825 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:03.873403 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25502dbf-f964-42ed-836a-4d090b0df825 None admin] PATH_INFO: `/v3/domains/default/users/96f47b71ea5a45ad92368f6b4130c36c/roles/4318f86ba9f74cd4a0bef1c311276657` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:03.873509 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6c3bc0-5018-4056-b1fb-43aa1b233879 None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:03.873720 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6c3bc0-5018-4056-b1fb-43aa1b233879 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:03.874032 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6c3bc0-5018-4056-b1fb-43aa1b233879 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:03.874788 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6c3bc0-5018-4056-b1fb-43aa1b233879 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:03.875480 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25502dbf-f964-42ed-836a-4d090b0df825 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=96f47b71ea5a45ad92368f6b4130c36c, role_id=4318f86ba9f74cd4a0bef1c311276657)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:03.876315 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6c3bc0-5018-4056-b1fb-43aa1b233879 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:03.876916 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25502dbf-f964-42ed-836a-4d090b0df825 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:03.955378 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 18/35] 10.222.0.33 () {60 vars in 1131 bytes} [Fri Jul 15 19:14:03 2022] POST /identity/v3/roles => generated 225 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:14:03.956141 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6c3bc0-5018-4056-b1fb-43aa1b233879 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:04.026177 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:04.079446 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6c3bc0-5018-4056-b1fb-43aa1b233879 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:04.118118 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4d32ce-a0e4-48ec-85b3-2af92147a4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:04.120565 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d32ce-a0e4-48ec-85b3-2af92147a4dd None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:04.120565 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d32ce-a0e4-48ec-85b3-2af92147a4dd None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:04.120565 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d32ce-a0e4-48ec-85b3-2af92147a4dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:04.130739 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 18/36] 10.222.0.33 () {58 vars in 1361 bytes} [Fri Jul 15 19:14:03 2022] PUT /identity/v3/domains/default/users/96f47b71ea5a45ad92368f6b4130c36c/roles/4318f86ba9f74cd4a0bef1c311276657 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:14:04.133457 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a608650-e5dd-42c1-99ac-a7803a42ffdb None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:04.133757 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a608650-e5dd-42c1-99ac-a7803a42ffdb None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:04.134077 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a608650-e5dd-42c1-99ac-a7803a42ffdb None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:04.137828 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4d32ce-a0e4-48ec-85b3-2af92147a4dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:04.138117 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4d32ce-a0e4-48ec-85b3-2af92147a4dd None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:04.140719 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 19/37] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:14:04 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:14:04.152287 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cee938-c6b8-4b83-aa3c-9fc830d1408b None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:04.152576 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cee938-c6b8-4b83-aa3c-9fc830d1408b None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:04.152858 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cee938-c6b8-4b83-aa3c-9fc830d1408b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:04.166552 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-0f4d32ce-a0e4-48ec-85b3-2af92147a4dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:14:04.166552 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:14:04.166552 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:14:04.166552 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:14:04.166552 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:14:04.166552 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:14:04.166552 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:14:04.166552 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:14:04.166552 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:14:04.166552 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:14:04.166552 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:14:04.166552 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:14:04.166552 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:14:04.166552 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:14:04.166552 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:14:04.167590 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:14:04.167590 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:14:04.167590 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:14:04.167590 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:14:04.167590 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:14:04.167590 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:14:04.167590 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:14:04.167590 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:14:04.167590 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:14:04.167590 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:14:04.167590 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:14:04.167590 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:14:04.167590 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:14:04.167590 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:14:04.167590 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:14:04.167590 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:14:04.167590 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:14:04.168621 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:14:04.168621 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:14:04.168621 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:14:04.168621 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:14:04.170266 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 19/38] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:14:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:14:04.176468 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-22cee938-c6b8-4b83-aa3c-9fc830d1408b None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:14:04.176793 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-22cee938-c6b8-4b83-aa3c-9fc830d1408b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:14:04.177981 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4d32ce-a0e4-48ec-85b3-2af92147a4dd None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:04.273904 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 20/39] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:14:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:14:04.273904 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25da01d8-9109-4c01-a9cd-6c0e7a3ed22a None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:04.273904 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25da01d8-9109-4c01-a9cd-6c0e7a3ed22a None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:04.273904 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25da01d8-9109-4c01-a9cd-6c0e7a3ed22a None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:04.273904 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 21/40] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:14:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:14:04.279511 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd0c981-2a1e-4d35-930f-cfab41954a6f None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:04.279806 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd0c981-2a1e-4d35-930f-cfab41954a6f None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:04.280094 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd0c981-2a1e-4d35-930f-cfab41954a6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:04.299279 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-8cd0c981-2a1e-4d35-930f-cfab41954a6f None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:14:04.300502 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-8cd0c981-2a1e-4d35-930f-cfab41954a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:14:04.313276 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4d32ce-a0e4-48ec-85b3-2af92147a4dd None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:04.320501 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:04.350865 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574ff12b-314c-4ddd-a5e9-18687112b895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:04.351530 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574ff12b-314c-4ddd-a5e9-18687112b895 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:04.351819 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574ff12b-314c-4ddd-a5e9-18687112b895 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:04.352098 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574ff12b-314c-4ddd-a5e9-18687112b895 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:04.352898 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574ff12b-314c-4ddd-a5e9-18687112b895 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:04.353206 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574ff12b-314c-4ddd-a5e9-18687112b895 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:04.354834 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574ff12b-314c-4ddd-a5e9-18687112b895 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:04.372824 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 20/41] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:14:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:04.373166 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 22/42] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:14:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 19:14:04.374095 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574ff12b-314c-4ddd-a5e9-18687112b895 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:04.383218 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd0c981-2a1e-4d35-930f-cfab41954a6f None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:04.411054 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:04.488867 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd0c981-2a1e-4d35-930f-cfab41954a6f None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:04.497823 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574ff12b-314c-4ddd-a5e9-18687112b895 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:04.526268 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0376d1-3a81-4b05-ad16-eaef64ae9ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:04.526968 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0376d1-3a81-4b05-ad16-eaef64ae9ab4 None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:04.527254 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0376d1-3a81-4b05-ad16-eaef64ae9ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:04.527530 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0376d1-3a81-4b05-ad16-eaef64ae9ab4 None admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:04.528244 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0376d1-3a81-4b05-ad16-eaef64ae9ab4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:04.530193 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0376d1-3a81-4b05-ad16-eaef64ae9ab4 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:04.534922 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0294ac-1cf0-4f20-9985-c3255b4ab2cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:04.535728 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0294ac-1cf0-4f20-9985-c3255b4ab2cf None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:04.536015 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0294ac-1cf0-4f20-9985-c3255b4ab2cf None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:04.536292 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0294ac-1cf0-4f20-9985-c3255b4ab2cf None admin] PATH_INFO: `/v3/domains/default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:04.537484 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0294ac-1cf0-4f20-9985-c3255b4ab2cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:04.538978 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0294ac-1cf0-4f20-9985-c3255b4ab2cf None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:04.541603 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 21/43] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:14:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:04.541936 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0294ac-1cf0-4f20-9985-c3255b4ab2cf None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:04.649817 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 23/44] 10.222.0.33 () {60 vars in 1141 bytes} [Fri Jul 15 19:14:04 2022] POST /identity/v3/projects => generated 301 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:14:04.649817 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0376d1-3a81-4b05-ad16-eaef64ae9ab4 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:04.705427 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:04.758089 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:04.791830 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0376d1-3a81-4b05-ad16-eaef64ae9ab4 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:04.796841 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0294ac-1cf0-4f20-9985-c3255b4ab2cf None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:04.847591 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2031358a-d9eb-41ce-9915-baa3f585e696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:04.848266 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2031358a-d9eb-41ce-9915-baa3f585e696 None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:04.848557 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2031358a-d9eb-41ce-9915-baa3f585e696 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:04.848837 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2031358a-d9eb-41ce-9915-baa3f585e696 None admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:04.849458 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eba5ed6-c460-49ea-9569-08fe195e5010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:04.849589 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2031358a-d9eb-41ce-9915-baa3f585e696 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:04.850457 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eba5ed6-c460-49ea-9569-08fe195e5010 None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:04.850947 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eba5ed6-c460-49ea-9569-08fe195e5010 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:04.851047 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2031358a-d9eb-41ce-9915-baa3f585e696 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:04.851502 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eba5ed6-c460-49ea-9569-08fe195e5010 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:04.852438 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eba5ed6-c460-49ea-9569-08fe195e5010 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:04.854083 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eba5ed6-c460-49ea-9569-08fe195e5010 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:04.949600 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 22/45] 10.222.0.33 () {60 vars in 1131 bytes} [Fri Jul 15 19:14:04 2022] POST /identity/v3/roles => generated 223 bytes in 408 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:14:04.985045 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 24/46] 10.222.0.33 () {60 vars in 1141 bytes} [Fri Jul 15 19:14:04 2022] POST /identity/v3/projects => generated 312 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:14:06.220516 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be761f3f-77b1-4c3c-978b-5d51f0ff309b None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:06.221007 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be761f3f-77b1-4c3c-978b-5d51f0ff309b None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:06.221007 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be761f3f-77b1-4c3c-978b-5d51f0ff309b None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:06.222410 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 23/47] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:14:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:14:06.227766 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2908f9-cb3d-4c4b-910a-70cca9be5d78 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:06.228180 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2908f9-cb3d-4c4b-910a-70cca9be5d78 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:06.228332 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2908f9-cb3d-4c4b-910a-70cca9be5d78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:06.244571 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-bb2908f9-cb3d-4c4b-910a-70cca9be5d78 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:14:06.245798 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-bb2908f9-cb3d-4c4b-910a-70cca9be5d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:14:06.311079 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 25/48] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:14:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:14:06.334563 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be761f3f-77b1-4c3c-978b-5d51f0ff309b None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:06.429564 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:06.437922 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be761f3f-77b1-4c3c-978b-5d51f0ff309b None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:06.470692 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063d7d1f-3858-4dce-9e0e-a55ce2c2fd8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:06.471416 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063d7d1f-3858-4dce-9e0e-a55ce2c2fd8b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:06.471702 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063d7d1f-3858-4dce-9e0e-a55ce2c2fd8b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:06.471979 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063d7d1f-3858-4dce-9e0e-a55ce2c2fd8b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:06.483281 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063d7d1f-3858-4dce-9e0e-a55ce2c2fd8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:14:06.490914 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-063d7d1f-3858-4dce-9e0e-a55ce2c2fd8b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:14:06.490914 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:14:06.490914 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:14:06.490914 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:14:06.490914 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:14:06.490914 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:14:06.490914 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:14:06.490914 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:14:06.490914 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:14:06.490914 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:14:06.490914 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:14:06.490914 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:14:06.490914 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:14:06.490914 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:14:06.490914 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:14:06.490914 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:14:06.491749 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:14:06.491749 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:14:06.491749 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:14:06.491749 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:14:06.491749 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:14:06.491749 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:14:06.491749 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:14:06.491749 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:14:06.491749 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:14:06.491749 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:14:06.491749 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:14:06.491749 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:14:06.491749 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:14:06.491749 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:14:06.491749 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:14:06.491749 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:14:06.492509 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:14:06.492509 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:14:06.492509 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:14:06.492509 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:14:06.492509 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:14:06.492509 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:14:06.492509 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:14:06.492509 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:14:06.492509 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:14:06.492509 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:14:06.492509 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:14:06.492509 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:14:06.492509 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 24/49] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jul 15 19:14:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:14:06.498548 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2908f9-cb3d-4c4b-910a-70cca9be5d78 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:06.556700 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:06.597827 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2908f9-cb3d-4c4b-910a-70cca9be5d78 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:06.630562 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7b9440-b926-4b2c-b759-e64973a3cc94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:06.631226 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7b9440-b926-4b2c-b759-e64973a3cc94 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:06.631508 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7b9440-b926-4b2c-b759-e64973a3cc94 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:06.631782 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7b9440-b926-4b2c-b759-e64973a3cc94 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:06.632529 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7b9440-b926-4b2c-b759-e64973a3cc94 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:06.632832 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7b9440-b926-4b2c-b759-e64973a3cc94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:06.634333 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7b9440-b926-4b2c-b759-e64973a3cc94 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:06.642497 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 26/50] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 15 19:14:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:14:06.648193 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063d7d1f-3858-4dce-9e0e-a55ce2c2fd8b None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:06.747002 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063d7d1f-3858-4dce-9e0e-a55ce2c2fd8b None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:06.780026 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c349d8-c3f0-458f-a1ef-d4dab2411c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:06.780906 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c349d8-c3f0-458f-a1ef-d4dab2411c43 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:06.781184 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c349d8-c3f0-458f-a1ef-d4dab2411c43 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:06.781456 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c349d8-c3f0-458f-a1ef-d4dab2411c43 None admin] PATH_INFO: `/v3/users/96f47b71ea5a45ad92368f6b4130c36c` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:06.782679 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c349d8-c3f0-458f-a1ef-d4dab2411c43 None admin] RBAC: Authorizing `identity:get_user(user_id=96f47b71ea5a45ad92368f6b4130c36c)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:06.784127 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c349d8-c3f0-458f-a1ef-d4dab2411c43 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:06.785952 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 25/51] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 15 19:14:06 2022] GET /identity/v3/users/96f47b71ea5a45ad92368f6b4130c36c => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:14:06.791861 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7b9440-b926-4b2c-b759-e64973a3cc94 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:06.996990 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7b9440-b926-4b2c-b759-e64973a3cc94 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:07.017005 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:07.031216 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9583cbd-171d-4f3a-8e7c-89d3d957de26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:07.032006 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9583cbd-171d-4f3a-8e7c-89d3d957de26 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:07.032287 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9583cbd-171d-4f3a-8e7c-89d3d957de26 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:07.032561 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9583cbd-171d-4f3a-8e7c-89d3d957de26 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:07.033867 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9583cbd-171d-4f3a-8e7c-89d3d957de26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:07.035354 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9583cbd-171d-4f3a-8e7c-89d3d957de26 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:07.036867 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab93296-30ec-4168-b4d8-e2a81f1765b7 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:07.036985 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 27/52] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:14:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:07.037173 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab93296-30ec-4168-b4d8-e2a81f1765b7 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:07.037462 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab93296-30ec-4168-b4d8-e2a81f1765b7 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:07.038871 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 26/53] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:14:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 19:14:07.043061 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9583cbd-171d-4f3a-8e7c-89d3d957de26 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:07.043916 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd53c389-5550-4b96-983c-2d24284b7d69 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:07.044205 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd53c389-5550-4b96-983c-2d24284b7d69 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:07.044486 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd53c389-5550-4b96-983c-2d24284b7d69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:07.060747 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-cd53c389-5550-4b96-983c-2d24284b7d69 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:14:07.061996 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-cd53c389-5550-4b96-983c-2d24284b7d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:14:07.079840 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:07.131075 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 27/54] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:14:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 19:14:07.141868 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9583cbd-171d-4f3a-8e7c-89d3d957de26 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:07.154947 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd53c389-5550-4b96-983c-2d24284b7d69 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:07.175664 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb9da18-43ad-49af-9185-1ff598b21d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:07.176328 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb9da18-43ad-49af-9185-1ff598b21d07 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:07.176620 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb9da18-43ad-49af-9185-1ff598b21d07 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:07.176905 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb9da18-43ad-49af-9185-1ff598b21d07 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:07.177751 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb9da18-43ad-49af-9185-1ff598b21d07 None admin] RBAC: Adding query filter params (role.id=4318f86ba9f74cd4a0bef1c311276657, scope.domain.id=default, user.id=96f47b71ea5a45ad92368f6b4130c36c) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:07.178093 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb9da18-43ad-49af-9185-1ff598b21d07 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:07.179533 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb9da18-43ad-49af-9185-1ff598b21d07 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:07.187963 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 28/55] 10.222.0.33 () {56 vars in 1322 bytes} [Fri Jul 15 19:14:07 2022] GET /identity/v3/role_assignments?user.id=96f47b71ea5a45ad92368f6b4130c36c&scope.domain.id=default&role.id=4318f86ba9f74cd4a0bef1c311276657 => generated 530 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:14:07.258099 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd53c389-5550-4b96-983c-2d24284b7d69 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:07.294457 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b419aae2-4d1a-4d49-a73a-2ed0f42ecdd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:07.295188 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b419aae2-4d1a-4d49-a73a-2ed0f42ecdd0 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:07.295476 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b419aae2-4d1a-4d49-a73a-2ed0f42ecdd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:07.295755 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b419aae2-4d1a-4d49-a73a-2ed0f42ecdd0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:07.297007 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b419aae2-4d1a-4d49-a73a-2ed0f42ecdd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:07.298518 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b419aae2-4d1a-4d49-a73a-2ed0f42ecdd0 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:07.300670 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 28/56] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:14:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:14:07.307001 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb9da18-43ad-49af-9185-1ff598b21d07 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:07.363374 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:07.411886 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb9da18-43ad-49af-9185-1ff598b21d07 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:07.445354 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b976f8-84c3-473b-ba2e-164b5c75b36a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:07.446052 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b976f8-84c3-473b-ba2e-164b5c75b36a None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:07.446349 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b976f8-84c3-473b-ba2e-164b5c75b36a None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:07.446623 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b976f8-84c3-473b-ba2e-164b5c75b36a None admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:07.447923 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b976f8-84c3-473b-ba2e-164b5c75b36a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:07.449384 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b976f8-84c3-473b-ba2e-164b5c75b36a None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:07.538006 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 29/57] 10.222.0.33 () {60 vars in 1140 bytes} [Fri Jul 15 19:14:07 2022] POST /identity/v3/projects => generated 298 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:14:09.560260 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b74522-0299-467e-b663-039e5c10cbce None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:09.560778 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b74522-0299-467e-b663-039e5c10cbce None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:09.560892 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b74522-0299-467e-b663-039e5c10cbce None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:09.562258 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 29/58] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:14:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:14:09.567581 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938ad108-57f6-4519-b51e-7893a94c332b None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:09.568024 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938ad108-57f6-4519-b51e-7893a94c332b None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:09.568152 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938ad108-57f6-4519-b51e-7893a94c332b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:09.584804 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-938ad108-57f6-4519-b51e-7893a94c332b None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:14:09.586050 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-938ad108-57f6-4519-b51e-7893a94c332b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:14:09.654140 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 30/59] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:14:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:14:09.678370 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b74522-0299-467e-b663-039e5c10cbce None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:09.700082 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:09.781944 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b74522-0299-467e-b663-039e5c10cbce None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:09.815360 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0961b5ba-2607-49d4-bd56-3067c81d3d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:09.816099 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0961b5ba-2607-49d4-bd56-3067c81d3d8a None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:09.816383 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0961b5ba-2607-49d4-bd56-3067c81d3d8a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:09.816660 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0961b5ba-2607-49d4-bd56-3067c81d3d8a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:09.817886 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0961b5ba-2607-49d4-bd56-3067c81d3d8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:09.819347 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0961b5ba-2607-49d4-bd56-3067c81d3d8a None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:09.820975 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 30/60] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:14:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:09.826650 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938ad108-57f6-4519-b51e-7893a94c332b None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:09.860020 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:09.924641 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938ad108-57f6-4519-b51e-7893a94c332b None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:09.958162 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dea757-6a34-47b0-9411-fd8e709b5ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:09.958828 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dea757-6a34-47b0-9411-fd8e709b5ac6 None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:09.959117 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dea757-6a34-47b0-9411-fd8e709b5ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:09.959398 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dea757-6a34-47b0-9411-fd8e709b5ac6 None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:09.960104 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dea757-6a34-47b0-9411-fd8e709b5ac6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:09.961560 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dea757-6a34-47b0-9411-fd8e709b5ac6 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:10.092033 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 31/61] 10.222.0.33 () {60 vars in 1132 bytes} [Fri Jul 15 19:14:09 2022] POST /identity/v3/users => generated 275 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:14:13.205140 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aac02a-b823-4395-8c9d-995c1cc3a646 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:13.205140 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aac02a-b823-4395-8c9d-995c1cc3a646 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:13.205140 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aac02a-b823-4395-8c9d-995c1cc3a646 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:13.205140 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 31/62] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:14:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:14:13.215339 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b480b0c-3a26-4f04-8dd4-54d1c8809c71 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:13.215872 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b480b0c-3a26-4f04-8dd4-54d1c8809c71 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:13.215949 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b480b0c-3a26-4f04-8dd4-54d1c8809c71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:13.242864 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-3b480b0c-3a26-4f04-8dd4-54d1c8809c71 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:14:13.244690 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-3b480b0c-3a26-4f04-8dd4-54d1c8809c71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:14:13.320521 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 32/63] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:14:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:14:13.356639 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aac02a-b823-4395-8c9d-995c1cc3a646 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:13.534283 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:13.547976 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aac02a-b823-4395-8c9d-995c1cc3a646 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:13.550779 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b7ab3b-044b-4032-9ac5-7f16e034016d None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:13.551083 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b7ab3b-044b-4032-9ac5-7f16e034016d None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:13.551363 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b7ab3b-044b-4032-9ac5-7f16e034016d None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:13.552904 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 33/64] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:14:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 19:14:13.558774 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57519cdc-f9a8-439b-88bf-2cecd7b949f4 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:13.559063 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57519cdc-f9a8-439b-88bf-2cecd7b949f4 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:13.559346 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57519cdc-f9a8-439b-88bf-2cecd7b949f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:13.594603 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-57519cdc-f9a8-439b-88bf-2cecd7b949f4 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:14:13.595820 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-57519cdc-f9a8-439b-88bf-2cecd7b949f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:14:13.622602 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:13.631081 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a2c032-1f59-4156-bbc5-c2255d709e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:13.634760 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a2c032-1f59-4156-bbc5-c2255d709e8b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:13.634946 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a2c032-1f59-4156-bbc5-c2255d709e8b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:13.641745 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a2c032-1f59-4156-bbc5-c2255d709e8b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:13.662533 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a2c032-1f59-4156-bbc5-c2255d709e8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:14:13.681184 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-42a2c032-1f59-4156-bbc5-c2255d709e8b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 19:14:13.681184 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:14:13.681184 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:14:13.681184 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:14:13.681184 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:14:13.681184 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:14:13.681184 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:14:13.681184 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:14:13.681184 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:14:13.681184 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:14:13.681184 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:14:13.681184 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:14:13.681184 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:14:13.681184 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:14:13.681184 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:14:13.681184 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:14:13.682166 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:14:13.682166 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:14:13.682166 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:14:13.682166 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:14:13.682166 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:14:13.682166 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:14:13.682166 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:14:13.682166 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:14:13.682166 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:14:13.682166 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:14:13.682166 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:14:13.682166 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:14:13.682166 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:14:13.682166 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:14:13.682166 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:14:13.682166 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:14:13.682952 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:14:13.682952 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:14:13.682952 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:14:13.682952 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:14:13.682952 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:14:13.682952 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:14:13.682952 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:14:13.682952 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:14:13.682952 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:14:13.682952 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:14:13.682952 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 19:14:13.682952 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:14:13.686890 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 32/65] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 15 19:14:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:14:13.688840 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a13c185-993a-4bcf-8cdb-294bf0480d3d None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:13.689136 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a13c185-993a-4bcf-8cdb-294bf0480d3d None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:13.689793 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a13c185-993a-4bcf-8cdb-294bf0480d3d None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:13.694716 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 33/66] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:14:13 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:14:13.696209 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a13c185-993a-4bcf-8cdb-294bf0480d3d None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:13.736316 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 34/67] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:14:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:14:13.740509 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc06226a-da6d-456b-95ed-337c026b2bdb None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:13.740509 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc06226a-da6d-456b-95ed-337c026b2bdb None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:13.740509 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc06226a-da6d-456b-95ed-337c026b2bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:13.797504 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-dc06226a-da6d-456b-95ed-337c026b2bdb None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:14:13.800832 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-dc06226a-da6d-456b-95ed-337c026b2bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:14:13.851196 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a13c185-993a-4bcf-8cdb-294bf0480d3d None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:13.874809 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 35/68] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:14:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:14:13.875923 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc06226a-da6d-456b-95ed-337c026b2bdb None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:13.896014 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3c7bda-a7fe-483b-9a72-e272f1fdc418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:13.897193 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3c7bda-a7fe-483b-9a72-e272f1fdc418 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:13.897493 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3c7bda-a7fe-483b-9a72-e272f1fdc418 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:13.897807 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3c7bda-a7fe-483b-9a72-e272f1fdc418 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:13.898562 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3c7bda-a7fe-483b-9a72-e272f1fdc418 None admin] RBAC: Adding query filter params (name=member) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:13.898876 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3c7bda-a7fe-483b-9a72-e272f1fdc418 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:13.901065 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3c7bda-a7fe-483b-9a72-e272f1fdc418 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:13.910038 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 34/69] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jul 15 19:14:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:13.911264 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed2c24c-fcf3-4307-a923-5c70cb92380c None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:13.911555 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed2c24c-fcf3-4307-a923-5c70cb92380c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:13.911836 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed2c24c-fcf3-4307-a923-5c70cb92380c None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:13.913073 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 35/70] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:14:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:14:13.917148 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0deb3a-da8c-4f2d-99e0-802d7c896145 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:13.917148 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0deb3a-da8c-4f2d-99e0-802d7c896145 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:13.917148 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0deb3a-da8c-4f2d-99e0-802d7c896145 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:13.917148 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 36/71] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:14:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:14:13.917148 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0deb3a-da8c-4f2d-99e0-802d7c896145 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:13.929861 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:13.961057 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:13.981203 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc06226a-da6d-456b-95ed-337c026b2bdb None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:14.020519 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0087655-66a8-4e1c-8893-45dba639145f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:14.020957 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0deb3a-da8c-4f2d-99e0-802d7c896145 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:14.021345 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0087655-66a8-4e1c-8893-45dba639145f None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:14.021700 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0087655-66a8-4e1c-8893-45dba639145f None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:14.022025 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0087655-66a8-4e1c-8893-45dba639145f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:14.033018 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0087655-66a8-4e1c-8893-45dba639145f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:14:14.041252 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-a0087655-66a8-4e1c-8893-45dba639145f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 19:14:14.041252 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:14:14.041252 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:14:14.041252 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:14:14.041252 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:14:14.041252 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:14:14.041252 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:14:14.041252 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:14:14.041252 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:14:14.041252 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:14:14.041252 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:14:14.041252 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:14:14.041252 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:14:14.041252 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:14:14.041252 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:14:14.041252 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:14:14.042297 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:14:14.042297 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:14:14.042297 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:14:14.042297 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:14:14.042297 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:14:14.042297 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:14:14.042297 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:14:14.042297 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:14:14.042297 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:14:14.042297 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:14:14.042297 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:14:14.042297 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:14:14.042297 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:14:14.042297 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:14:14.042297 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:14:14.042297 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:14:14.043253 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:14:14.043253 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:14:14.043253 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:14:14.043253 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:14:14.043253 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:14:14.043253 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:14:14.043253 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:14:14.043253 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:14:14.043253 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:14:14.043253 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:14:14.043253 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 19:14:14.043253 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:14:14.043253 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 36/72] 10.222.0.33 () {56 vars in 1114 bytes} [Fri Jul 15 19:14:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:14:14.044085 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0087655-66a8-4e1c-8893-45dba639145f None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:14.059836 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e677c1-6b4d-4d37-b039-2cc9ff4d306b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:14.060527 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e677c1-6b4d-4d37-b039-2cc9ff4d306b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:14.060812 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e677c1-6b4d-4d37-b039-2cc9ff4d306b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:14.061082 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e677c1-6b4d-4d37-b039-2cc9ff4d306b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:14.073558 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e677c1-6b4d-4d37-b039-2cc9ff4d306b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:14:14.081302 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-02e677c1-6b4d-4d37-b039-2cc9ff4d306b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:14:14.081302 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:14:14.081302 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:14:14.081302 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:14:14.081302 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:14:14.081302 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:14:14.081302 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:14:14.081302 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:14:14.081302 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:14:14.081302 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:14:14.081302 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:14:14.081302 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:14:14.081302 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:14:14.081302 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:14:14.081302 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:14:14.081302 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:14:14.082217 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:14:14.082217 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:14:14.082217 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:14:14.082217 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:14:14.082217 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:14:14.082217 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:14:14.082217 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:14:14.082217 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:14:14.082217 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:14:14.082217 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:14:14.082217 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:14:14.082217 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:14:14.082217 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:14:14.082217 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:14:14.082217 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:14:14.082217 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:14:14.082996 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:14:14.082996 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:14:14.082996 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:14:14.082996 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:14:14.082996 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:14:14.082996 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:14:14.082996 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:14:14.082996 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:14:14.082996 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:14:14.082996 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:14:14.082996 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:14:14.082996 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:14:14.082996 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 37/73] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jul 15 19:14:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:14:14.084107 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be233fb-fa34-40b0-abb3-298bf7c4cfa7 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:14.084107 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be233fb-fa34-40b0-abb3-298bf7c4cfa7 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:14.084280 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be233fb-fa34-40b0-abb3-298bf7c4cfa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:14.100296 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-6be233fb-fa34-40b0-abb3-298bf7c4cfa7 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:14:14.101486 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-6be233fb-fa34-40b0-abb3-298bf7c4cfa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:14:14.147933 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0087655-66a8-4e1c-8893-45dba639145f None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:14.164396 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:14.169358 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 38/74] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:14:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:14:14.175675 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2285e0c-225f-45b0-9191-0b1a5b99b333 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:14.182375 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2285e0c-225f-45b0-9191-0b1a5b99b333 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:14.182375 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2285e0c-225f-45b0-9191-0b1a5b99b333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:14.183098 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fec49b-56a6-4357-810e-03c779ff8a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:14.183983 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fec49b-56a6-4357-810e-03c779ff8a94 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:14.184272 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fec49b-56a6-4357-810e-03c779ff8a94 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:14.184552 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fec49b-56a6-4357-810e-03c779ff8a94 None admin] PATH_INFO: `/v3/users/da619d0bd9ad4a48a65b481207b21979` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:14.193266 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-c2285e0c-225f-45b0-9191-0b1a5b99b333 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:14:14.194496 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-c2285e0c-225f-45b0-9191-0b1a5b99b333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:14:14.195761 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:14.201454 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fec49b-56a6-4357-810e-03c779ff8a94 None admin] RBAC: Authorizing `identity:get_user(user_id=da619d0bd9ad4a48a65b481207b21979)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:14.203044 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fec49b-56a6-4357-810e-03c779ff8a94 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:14.211201 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 37/75] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 15 19:14:14 2022] GET /identity/v3/users/da619d0bd9ad4a48a65b481207b21979 => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:14.211201 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fec49b-56a6-4357-810e-03c779ff8a94 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:14.267313 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 39/76] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:14:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:14:14.268186 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2285e0c-225f-45b0-9191-0b1a5b99b333 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:14.316316 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fec49b-56a6-4357-810e-03c779ff8a94 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:14.350296 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf161d1f-bad4-47f5-9e2d-17654e4c1cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:14.350965 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf161d1f-bad4-47f5-9e2d-17654e4c1cfd None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:14.351251 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf161d1f-bad4-47f5-9e2d-17654e4c1cfd None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:14.351528 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf161d1f-bad4-47f5-9e2d-17654e4c1cfd None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:14.352282 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf161d1f-bad4-47f5-9e2d-17654e4c1cfd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:14.352598 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf161d1f-bad4-47f5-9e2d-17654e4c1cfd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:14.354529 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf161d1f-bad4-47f5-9e2d-17654e4c1cfd None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:14.362337 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 38/77] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 15 19:14:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:14.363109 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf161d1f-bad4-47f5-9e2d-17654e4c1cfd None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:14.371519 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2285e0c-225f-45b0-9191-0b1a5b99b333 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:14.399218 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:14.405197 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726ea455-1a09-470e-84ab-8f92298061a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:14.405898 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726ea455-1a09-470e-84ab-8f92298061a6 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:14.406186 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726ea455-1a09-470e-84ab-8f92298061a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:14.406470 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726ea455-1a09-470e-84ab-8f92298061a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:14.407220 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726ea455-1a09-470e-84ab-8f92298061a6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:14.407523 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726ea455-1a09-470e-84ab-8f92298061a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:14.408923 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726ea455-1a09-470e-84ab-8f92298061a6 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:14.417389 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 40/78] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 15 19:14:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:14.422229 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726ea455-1a09-470e-84ab-8f92298061a6 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:14.427399 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:14.467466 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf161d1f-bad4-47f5-9e2d-17654e4c1cfd None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:14.500725 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee5bfa6-bf1c-430a-a172-bd2bfe5fdc91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:14.501513 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee5bfa6-bf1c-430a-a172-bd2bfe5fdc91 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:14.501830 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee5bfa6-bf1c-430a-a172-bd2bfe5fdc91 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:14.502113 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee5bfa6-bf1c-430a-a172-bd2bfe5fdc91 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:14.513067 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee5bfa6-bf1c-430a-a172-bd2bfe5fdc91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:14:14.516843 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726ea455-1a09-470e-84ab-8f92298061a6 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:14.520708 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-0ee5bfa6-bf1c-430a-a172-bd2bfe5fdc91 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 19:14:14.520708 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:14:14.520708 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:14:14.520708 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:14:14.520708 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:14:14.520708 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:14:14.520708 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:14:14.520708 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:14:14.520708 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:14:14.520708 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:14:14.520708 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:14:14.520708 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:14:14.520708 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:14:14.520708 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:14:14.520708 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:14:14.520708 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:14:14.521576 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:14:14.521576 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:14:14.521576 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:14:14.521576 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:14:14.521576 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:14:14.521576 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:14:14.521576 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:14:14.521576 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:14:14.521576 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:14:14.521576 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:14:14.521576 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:14:14.521576 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:14:14.521576 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:14:14.521576 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:14:14.521576 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:14:14.521576 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:14:14.522410 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:14:14.522410 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:14:14.522410 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:14:14.522410 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:14:14.522410 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:14:14.522410 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:14:14.522410 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:14:14.522410 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:14:14.522410 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:14:14.522410 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:14:14.522410 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 19:14:14.522410 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:14:14.522410 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 39/79] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 15 19:14:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:14:14.523053 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee5bfa6-bf1c-430a-a172-bd2bfe5fdc91 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:14.551935 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57cd38b-e3ba-47ef-bf3b-4d15b8a1090b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:14.552580 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57cd38b-e3ba-47ef-bf3b-4d15b8a1090b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:14.552860 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57cd38b-e3ba-47ef-bf3b-4d15b8a1090b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:14.553138 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57cd38b-e3ba-47ef-bf3b-4d15b8a1090b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:14.615807 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:14.624319 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee5bfa6-bf1c-430a-a172-bd2bfe5fdc91 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:14.626037 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57cd38b-e3ba-47ef-bf3b-4d15b8a1090b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:14.627554 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57cd38b-e3ba-47ef-bf3b-4d15b8a1090b None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:14.643735 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:14.657526 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c719b6-4d1f-458a-b05c-3cda17499f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:14.658360 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c719b6-4d1f-458a-b05c-3cda17499f52 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:14.658643 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c719b6-4d1f-458a-b05c-3cda17499f52 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:14.662148 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c719b6-4d1f-458a-b05c-3cda17499f52 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:14.662148 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c719b6-4d1f-458a-b05c-3cda17499f52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:14.662148 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c719b6-4d1f-458a-b05c-3cda17499f52 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:14.663156 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 40/80] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:14:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:14.663914 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c719b6-4d1f-458a-b05c-3cda17499f52 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:14.732055 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 41/81] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 15 19:14:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 19:14:14.732977 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57cd38b-e3ba-47ef-bf3b-4d15b8a1090b None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:14.766833 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c719b6-4d1f-458a-b05c-3cda17499f52 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:14.802268 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816fd53e-6ff3-44a6-a079-2da218ef7ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:14.803386 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816fd53e-6ff3-44a6-a079-2da218ef7ed5 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:14.803675 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816fd53e-6ff3-44a6-a079-2da218ef7ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:14.803955 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816fd53e-6ff3-44a6-a079-2da218ef7ed5 None admin] PATH_INFO: `/v3/users/da619d0bd9ad4a48a65b481207b21979` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:14.805185 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816fd53e-6ff3-44a6-a079-2da218ef7ed5 None admin] RBAC: Authorizing `identity:get_user(user_id=da619d0bd9ad4a48a65b481207b21979)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:14.807901 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816fd53e-6ff3-44a6-a079-2da218ef7ed5 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:14.810180 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 41/82] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 15 19:14:14 2022] GET /identity/v3/users/da619d0bd9ad4a48a65b481207b21979 => generated 275 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:14.810755 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816fd53e-6ff3-44a6-a079-2da218ef7ed5 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:14.827774 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:14.835949 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57cd38b-e3ba-47ef-bf3b-4d15b8a1090b None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:14.867090 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:14.869893 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ab39a-664d-4d15-9563-3d6306dde9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:14.870769 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ab39a-664d-4d15-9563-3d6306dde9fc None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:14.871044 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ab39a-664d-4d15-9563-3d6306dde9fc None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:14.871316 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ab39a-664d-4d15-9563-3d6306dde9fc None admin] PATH_INFO: `/v3/users/96f47b71ea5a45ad92368f6b4130c36c` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:14.872510 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4ab39a-664d-4d15-9563-3d6306dde9fc None admin] RBAC: Authorizing `identity:get_user(user_id=96f47b71ea5a45ad92368f6b4130c36c)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:14.873995 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4ab39a-664d-4d15-9563-3d6306dde9fc None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:14.875633 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 42/83] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 15 19:14:14 2022] GET /identity/v3/users/96f47b71ea5a45ad92368f6b4130c36c => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:14.876392 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ab39a-664d-4d15-9563-3d6306dde9fc None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:14.910978 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816fd53e-6ff3-44a6-a079-2da218ef7ed5 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:14.957217 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd78765-4ef5-408d-8705-e9d0eccc1463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:14.957936 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd78765-4ef5-408d-8705-e9d0eccc1463 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:14.958219 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd78765-4ef5-408d-8705-e9d0eccc1463 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:14.958490 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd78765-4ef5-408d-8705-e9d0eccc1463 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:14.959245 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd78765-4ef5-408d-8705-e9d0eccc1463 None admin] RBAC: Adding query filter params (name=member) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:14.959549 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd78765-4ef5-408d-8705-e9d0eccc1463 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:14.960929 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd78765-4ef5-408d-8705-e9d0eccc1463 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:14.969783 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 42/84] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jul 15 19:14:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:14.974712 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd78765-4ef5-408d-8705-e9d0eccc1463 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:14.979021 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ab39a-664d-4d15-9563-3d6306dde9fc None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:15.013751 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12abf2b2-6c06-485d-b4b3-2ade0d4a82db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:15.014445 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12abf2b2-6c06-485d-b4b3-2ade0d4a82db None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:15.014735 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12abf2b2-6c06-485d-b4b3-2ade0d4a82db None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:15.015014 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12abf2b2-6c06-485d-b4b3-2ade0d4a82db None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:15.015735 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12abf2b2-6c06-485d-b4b3-2ade0d4a82db None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:15.017202 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12abf2b2-6c06-485d-b4b3-2ade0d4a82db None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:15.069834 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd78765-4ef5-408d-8705-e9d0eccc1463 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:15.085769 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:15.102116 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c04a512-2961-4c30-86e1-c84e517d55cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:15.103507 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c04a512-2961-4c30-86e1-c84e517d55cb None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:15.103911 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c04a512-2961-4c30-86e1-c84e517d55cb None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:15.104188 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c04a512-2961-4c30-86e1-c84e517d55cb None admin] PATH_INFO: `/v3/projects/513641c037c84c54b779b5ca0684f17c` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:15.105370 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c04a512-2961-4c30-86e1-c84e517d55cb None admin] RBAC: Authorizing `identity:get_project(project_id=513641c037c84c54b779b5ca0684f17c)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:15.106828 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c04a512-2961-4c30-86e1-c84e517d55cb None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:15.108520 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 43/85] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 15 19:14:14 2022] GET /identity/v3/projects/513641c037c84c54b779b5ca0684f17c => generated 298 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:15.109563 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c04a512-2961-4c30-86e1-c84e517d55cb None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:15.319700 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c04a512-2961-4c30-86e1-c84e517d55cb None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:15.319700 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf4123e-a3b0-4cac-9f6e-4da00254d0a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:15.319700 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf4123e-a3b0-4cac-9f6e-4da00254d0a8 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:15.319700 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf4123e-a3b0-4cac-9f6e-4da00254d0a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:15.320540 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf4123e-a3b0-4cac-9f6e-4da00254d0a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:15.320540 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf4123e-a3b0-4cac-9f6e-4da00254d0a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:15.320540 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf4123e-a3b0-4cac-9f6e-4da00254d0a8 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:15.392726 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 44/86] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 15 19:14:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 19:14:15.393527 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf4123e-a3b0-4cac-9f6e-4da00254d0a8 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:15.486231 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 43/87] 10.222.0.33 () {60 vars in 1132 bytes} [Fri Jul 15 19:14:14 2022] POST /identity/v3/users => generated 289 bytes in 610 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:14:15.486860 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf4123e-a3b0-4cac-9f6e-4da00254d0a8 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:15.487061 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12abf2b2-6c06-485d-b4b3-2ade0d4a82db None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:15.491523 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:15.542439 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:15.560738 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d9defa-07d3-41dc-9152-8b3f0b3db796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:15.561409 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d9defa-07d3-41dc-9152-8b3f0b3db796 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:15.561695 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d9defa-07d3-41dc-9152-8b3f0b3db796 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:15.562005 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d9defa-07d3-41dc-9152-8b3f0b3db796 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:15.607066 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:15.634817 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12abf2b2-6c06-485d-b4b3-2ade0d4a82db None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:15.637369 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d9defa-07d3-41dc-9152-8b3f0b3db796 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:15.638893 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d9defa-07d3-41dc-9152-8b3f0b3db796 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:15.674035 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46624e3d-66c5-4d3a-a623-50d34d3afd16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:15.674931 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46624e3d-66c5-4d3a-a623-50d34d3afd16 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:15.675225 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46624e3d-66c5-4d3a-a623-50d34d3afd16 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:15.675504 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46624e3d-66c5-4d3a-a623-50d34d3afd16 None admin] PATH_INFO: `/v3/users/da619d0bd9ad4a48a65b481207b21979` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:15.676746 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46624e3d-66c5-4d3a-a623-50d34d3afd16 None admin] RBAC: Authorizing `identity:get_user(user_id=da619d0bd9ad4a48a65b481207b21979)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:15.678330 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46624e3d-66c5-4d3a-a623-50d34d3afd16 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:15.679993 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 44/88] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 15 19:14:15 2022] GET /identity/v3/users/da619d0bd9ad4a48a65b481207b21979 => generated 275 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:15.680881 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46624e3d-66c5-4d3a-a623-50d34d3afd16 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:15.745465 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:15.764097 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 45/89] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 15 19:14:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 19:14:15.764971 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d9defa-07d3-41dc-9152-8b3f0b3db796 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:15.801860 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:15.830648 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46624e3d-66c5-4d3a-a623-50d34d3afd16 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:15.868170 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df000b3-f847-49ba-9ab7-87cbc5f43d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:15.869146 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df000b3-f847-49ba-9ab7-87cbc5f43d6a None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:15.869609 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df000b3-f847-49ba-9ab7-87cbc5f43d6a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:15.870107 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df000b3-f847-49ba-9ab7-87cbc5f43d6a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:15.871131 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df000b3-f847-49ba-9ab7-87cbc5f43d6a None admin] RBAC: Adding query filter params (role.id=28763c6e70094116b3a4b2895895fa58, scope.project.id=513641c037c84c54b779b5ca0684f17c, user.id=da619d0bd9ad4a48a65b481207b21979) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:15.871620 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df000b3-f847-49ba-9ab7-87cbc5f43d6a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:15.873267 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df000b3-f847-49ba-9ab7-87cbc5f43d6a None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:15.879189 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 45/90] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 15 19:14:15 2022] GET /identity/v3/role_assignments?user.id=da619d0bd9ad4a48a65b481207b21979&scope.project.id=513641c037c84c54b779b5ca0684f17c&role.id=28763c6e70094116b3a4b2895895fa58 => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:15.880250 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df000b3-f847-49ba-9ab7-87cbc5f43d6a None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:15.905066 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d9defa-07d3-41dc-9152-8b3f0b3db796 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:15.964453 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2523641-d350-4fda-aac2-f6e6e8ddd56e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:15.965346 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2523641-d350-4fda-aac2-f6e6e8ddd56e None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:15.965631 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2523641-d350-4fda-aac2-f6e6e8ddd56e None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:15.965955 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2523641-d350-4fda-aac2-f6e6e8ddd56e None admin] PATH_INFO: `/v3/projects/513641c037c84c54b779b5ca0684f17c` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:15.967482 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2523641-d350-4fda-aac2-f6e6e8ddd56e None admin] RBAC: Authorizing `identity:get_project(project_id=513641c037c84c54b779b5ca0684f17c)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:15.968973 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2523641-d350-4fda-aac2-f6e6e8ddd56e None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:15.979809 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 46/91] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 15 19:14:15 2022] GET /identity/v3/projects/513641c037c84c54b779b5ca0684f17c => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:15.980808 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2523641-d350-4fda-aac2-f6e6e8ddd56e None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:16.036716 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df000b3-f847-49ba-9ab7-87cbc5f43d6a None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:16.088169 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359fae26-62a2-4a0c-87f4-75d032dcbfd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:16.088890 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359fae26-62a2-4a0c-87f4-75d032dcbfd3 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:16.089181 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359fae26-62a2-4a0c-87f4-75d032dcbfd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:16.089464 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359fae26-62a2-4a0c-87f4-75d032dcbfd3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:16.117015 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:16.118551 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:16.184952 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2523641-d350-4fda-aac2-f6e6e8ddd56e None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:16.231507 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be488f1-d649-4729-b506-e6cf25248480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:16.232394 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be488f1-d649-4729-b506-e6cf25248480 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:16.232683 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be488f1-d649-4729-b506-e6cf25248480 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:16.232966 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be488f1-d649-4729-b506-e6cf25248480 None admin] PATH_INFO: `/v3/projects/513641c037c84c54b779b5ca0684f17c` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:16.236738 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be488f1-d649-4729-b506-e6cf25248480 None admin] RBAC: Authorizing `identity:get_project(project_id=513641c037c84c54b779b5ca0684f17c)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:16.238226 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be488f1-d649-4729-b506-e6cf25248480 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:16.240157 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 47/92] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 15 19:14:15 2022] GET /identity/v3/projects/513641c037c84c54b779b5ca0684f17c => generated 298 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:16.240764 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be488f1-d649-4729-b506-e6cf25248480 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:16.245026 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359fae26-62a2-4a0c-87f4-75d032dcbfd3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:16.246477 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359fae26-62a2-4a0c-87f4-75d032dcbfd3 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:16.404236 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 46/93] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 15 19:14:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 19:14:16.405039 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359fae26-62a2-4a0c-87f4-75d032dcbfd3 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:16.435670 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be488f1-d649-4729-b506-e6cf25248480 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:16.456239 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:16.461984 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:16.491072 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a9542e-46f3-416a-b8e7-3be3370792d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:16.491744 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a9542e-46f3-416a-b8e7-3be3370792d8 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:16.492040 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a9542e-46f3-416a-b8e7-3be3370792d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:16.492319 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a9542e-46f3-416a-b8e7-3be3370792d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:16.493171 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a9542e-46f3-416a-b8e7-3be3370792d8 None admin] RBAC: Adding query filter params (role.id=e97d26621445466d9550261c608db9c5, scope.project.id=513641c037c84c54b779b5ca0684f17c, user.id=da619d0bd9ad4a48a65b481207b21979) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:16.494032 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a9542e-46f3-416a-b8e7-3be3370792d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:16.495518 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a9542e-46f3-416a-b8e7-3be3370792d8 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:16.510738 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359fae26-62a2-4a0c-87f4-75d032dcbfd3 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:16.515952 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 48/94] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 15 19:14:16 2022] GET /identity/v3/role_assignments?user.id=da619d0bd9ad4a48a65b481207b21979&scope.project.id=513641c037c84c54b779b5ca0684f17c&role.id=e97d26621445466d9550261c608db9c5 => generated 259 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:16.516781 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a9542e-46f3-416a-b8e7-3be3370792d8 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:16.570560 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28641070-45a5-428c-9edb-1651406275db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:16.571259 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28641070-45a5-428c-9edb-1651406275db None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:16.571610 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28641070-45a5-428c-9edb-1651406275db None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:16.571896 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28641070-45a5-428c-9edb-1651406275db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:16.572768 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28641070-45a5-428c-9edb-1651406275db None admin] RBAC: Adding query filter params (role.id=4318f86ba9f74cd4a0bef1c311276657, scope.project.id=513641c037c84c54b779b5ca0684f17c, user.id=96f47b71ea5a45ad92368f6b4130c36c) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:16.573088 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28641070-45a5-428c-9edb-1651406275db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:16.574720 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28641070-45a5-428c-9edb-1651406275db None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:16.580998 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 47/95] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 15 19:14:16 2022] GET /identity/v3/role_assignments?user.id=96f47b71ea5a45ad92368f6b4130c36c&scope.project.id=513641c037c84c54b779b5ca0684f17c&role.id=4318f86ba9f74cd4a0bef1c311276657 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:16.698981 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a9542e-46f3-416a-b8e7-3be3370792d8 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:16.749713 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1954f35c-87d1-4996-9bf7-698a917514c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:16.750616 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1954f35c-87d1-4996-9bf7-698a917514c4 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:16.750906 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1954f35c-87d1-4996-9bf7-698a917514c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:16.751189 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1954f35c-87d1-4996-9bf7-698a917514c4 None admin] PATH_INFO: `/v3/projects/e733f93067e74ac193989bcd00d8d5ce` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:16.758799 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1954f35c-87d1-4996-9bf7-698a917514c4 None admin] RBAC: Authorizing `identity:get_project(project_id=e733f93067e74ac193989bcd00d8d5ce)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:16.760421 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1954f35c-87d1-4996-9bf7-698a917514c4 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:16.762293 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 49/96] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 15 19:14:16 2022] GET /identity/v3/projects/e733f93067e74ac193989bcd00d8d5ce => generated 312 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:16.768609 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28641070-45a5-428c-9edb-1651406275db None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:16.904524 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28641070-45a5-428c-9edb-1651406275db None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:16.923450 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:16.956118 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addb9b61-cecc-4604-a6ae-7c5d275b097c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:16.960222 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addb9b61-cecc-4604-a6ae-7c5d275b097c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:16.960512 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addb9b61-cecc-4604-a6ae-7c5d275b097c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:16.960806 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addb9b61-cecc-4604-a6ae-7c5d275b097c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:16.961911 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addb9b61-cecc-4604-a6ae-7c5d275b097c None admin] RBAC: Adding query filter params (role.id=28763c6e70094116b3a4b2895895fa58, scope.project.id=e733f93067e74ac193989bcd00d8d5ce, user.id=da619d0bd9ad4a48a65b481207b21979) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:16.962513 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addb9b61-cecc-4604-a6ae-7c5d275b097c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:16.964674 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addb9b61-cecc-4604-a6ae-7c5d275b097c None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:16.974690 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 48/97] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 15 19:14:16 2022] GET /identity/v3/role_assignments?user.id=da619d0bd9ad4a48a65b481207b21979&scope.project.id=e733f93067e74ac193989bcd00d8d5ce&role.id=28763c6e70094116b3a4b2895895fa58 => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:18.041445 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553f05c7-b537-44b3-aa20-0cfa86e2e516 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:18.042509 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553f05c7-b537-44b3-aa20-0cfa86e2e516 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:18.043089 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553f05c7-b537-44b3-aa20-0cfa86e2e516 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:18.044735 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 50/98] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:14:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:14:18.063528 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cec75a-24ee-4c33-b723-5305f8fe3b81 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:18.064421 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cec75a-24ee-4c33-b723-5305f8fe3b81 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:18.064905 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cec75a-24ee-4c33-b723-5305f8fe3b81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:18.090755 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-14cec75a-24ee-4c33-b723-5305f8fe3b81 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:14:18.092320 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-14cec75a-24ee-4c33-b723-5305f8fe3b81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:14:18.178425 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 49/99] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:14:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:14:18.230063 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553f05c7-b537-44b3-aa20-0cfa86e2e516 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:18.283111 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1550194-658a-4d67-8cdb-f9c8b5633e75 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:18.283772 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1550194-658a-4d67-8cdb-f9c8b5633e75 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:18.284299 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1550194-658a-4d67-8cdb-f9c8b5633e75 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:18.285898 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 50/100] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:14:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:14:18.291262 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716e1e7a-1b0d-4eb0-98bb-4f3ca29ab374 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:18.291814 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716e1e7a-1b0d-4eb0-98bb-4f3ca29ab374 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:18.292313 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716e1e7a-1b0d-4eb0-98bb-4f3ca29ab374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:18.303614 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:18.326938 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-716e1e7a-1b0d-4eb0-98bb-4f3ca29ab374 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:14:18.328116 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-716e1e7a-1b0d-4eb0-98bb-4f3ca29ab374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:14:18.409004 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553f05c7-b537-44b3-aa20-0cfa86e2e516 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:18.440329 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 51/101] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:14:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:14:18.480568 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dd7584-0691-4560-b566-eaf35c0e2a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:18.480799 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716e1e7a-1b0d-4eb0-98bb-4f3ca29ab374 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:18.481389 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dd7584-0691-4560-b566-eaf35c0e2a44 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:18.481679 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dd7584-0691-4560-b566-eaf35c0e2a44 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:18.481998 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dd7584-0691-4560-b566-eaf35c0e2a44 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:18.486519 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dd7584-0691-4560-b566-eaf35c0e2a44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:18.488023 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dd7584-0691-4560-b566-eaf35c0e2a44 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:18.489720 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 51/102] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:14:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:18.497040 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dd7584-0691-4560-b566-eaf35c0e2a44 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:18.534741 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:18.636774 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716e1e7a-1b0d-4eb0-98bb-4f3ca29ab374 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:18.689414 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dd7584-0691-4560-b566-eaf35c0e2a44 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:18.707245 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d861004-eb43-483c-a296-9427efc39e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:18.708065 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d861004-eb43-483c-a296-9427efc39e8e None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:18.708342 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d861004-eb43-483c-a296-9427efc39e8e None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:18.708619 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d861004-eb43-483c-a296-9427efc39e8e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:18.731032 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d861004-eb43-483c-a296-9427efc39e8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:14:18.737852 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:18.746860 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-6d861004-eb43-483c-a296-9427efc39e8e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 19:14:18.746860 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:14:18.746860 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:14:18.746860 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:14:18.746860 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:14:18.746860 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:14:18.746860 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:14:18.746860 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:14:18.746860 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:14:18.746860 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:14:18.746860 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:14:18.746860 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:14:18.746860 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:14:18.746860 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:14:18.746860 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:14:18.746860 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:14:18.753736 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:14:18.753736 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:14:18.753736 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:14:18.753736 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:14:18.753736 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:14:18.753736 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:14:18.753736 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:14:18.753736 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:14:18.753736 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:14:18.753736 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:14:18.753736 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:14:18.753736 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:14:18.753736 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:14:18.753736 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:14:18.753736 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:14:18.753736 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:14:18.754577 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:14:18.754577 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:14:18.754577 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:14:18.754577 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:14:18.754577 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:14:18.754577 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:14:18.754577 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:14:18.754577 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:14:18.754577 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:14:18.754577 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:14:18.754577 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 19:14:18.754577 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:14:18.754577 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 52/103] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 15 19:14:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:14:18.754577 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d861004-eb43-483c-a296-9427efc39e8e None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:18.764863 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a70503-0931-4f3c-8e53-2a72482cbe20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:18.766025 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a70503-0931-4f3c-8e53-2a72482cbe20 None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:18.766528 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a70503-0931-4f3c-8e53-2a72482cbe20 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:18.767012 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a70503-0931-4f3c-8e53-2a72482cbe20 None admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:18.767954 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a70503-0931-4f3c-8e53-2a72482cbe20 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:18.769596 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a70503-0931-4f3c-8e53-2a72482cbe20 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:18.880496 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 52/104] 10.222.0.33 () {60 vars in 1141 bytes} [Fri Jul 15 19:14:18 2022] POST /identity/v3/projects => generated 302 bytes in 384 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:14:18.945703 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:18.951912 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d861004-eb43-483c-a296-9427efc39e8e None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:19.007598 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7116555-1cbd-4fd2-9bae-557af15eca7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:19.008292 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7116555-1cbd-4fd2-9bae-557af15eca7d None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:19.008579 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7116555-1cbd-4fd2-9bae-557af15eca7d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:19.008891 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7116555-1cbd-4fd2-9bae-557af15eca7d None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:19.009674 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7116555-1cbd-4fd2-9bae-557af15eca7d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:19.010005 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7116555-1cbd-4fd2-9bae-557af15eca7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:19.011415 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7116555-1cbd-4fd2-9bae-557af15eca7d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:19.025850 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 53/105] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jul 15 19:14:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:14:19.031682 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a70503-0931-4f3c-8e53-2a72482cbe20 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:19.079087 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d1a2c3-c719-4a2f-a0b8-66f07072ddf8 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:19.080109 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d1a2c3-c719-4a2f-a0b8-66f07072ddf8 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:19.080610 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d1a2c3-c719-4a2f-a0b8-66f07072ddf8 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:19.082299 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 54/106] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:14:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:14:19.088292 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1469c2-d992-41d3-bd0f-868055077c1e None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:19.088857 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1469c2-d992-41d3-bd0f-868055077c1e None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:19.089362 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1469c2-d992-41d3-bd0f-868055077c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:19.134937 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-7d1469c2-d992-41d3-bd0f-868055077c1e None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:14:19.136535 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-7d1469c2-d992-41d3-bd0f-868055077c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:14:19.182462 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a70503-0931-4f3c-8e53-2a72482cbe20 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:19.221494 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877e1290-0c7f-4391-827c-1252394c9384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:19.222836 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877e1290-0c7f-4391-827c-1252394c9384 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:19.223305 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877e1290-0c7f-4391-827c-1252394c9384 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:19.223746 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877e1290-0c7f-4391-827c-1252394c9384 None admin] PATH_INFO: `/v3/users/0505801f23f1420eb449bad3d0b36bec` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:19.242740 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877e1290-0c7f-4391-827c-1252394c9384 None admin] RBAC: Authorizing `identity:get_user(user_id=0505801f23f1420eb449bad3d0b36bec)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:19.245160 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877e1290-0c7f-4391-827c-1252394c9384 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:19.247650 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 53/107] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 15 19:14:19 2022] GET /identity/v3/users/0505801f23f1420eb449bad3d0b36bec => generated 289 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:19.254949 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877e1290-0c7f-4391-827c-1252394c9384 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:19.286537 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 55/108] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:14:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 19:14:19.296281 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:19.323216 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1469c2-d992-41d3-bd0f-868055077c1e None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:19.487856 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877e1290-0c7f-4391-827c-1252394c9384 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:19.557985 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:19.574523 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4f4c3a-1143-4179-bda4-201b2dfee72b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:19.575209 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f4c3a-1143-4179-bda4-201b2dfee72b None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:19.575500 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f4c3a-1143-4179-bda4-201b2dfee72b None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:19.575786 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f4c3a-1143-4179-bda4-201b2dfee72b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:19.579052 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1469c2-d992-41d3-bd0f-868055077c1e None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:19.654561 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c65f019-1f72-4024-a25f-c3f91e761b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:19.655388 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c65f019-1f72-4024-a25f-c3f91e761b46 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:19.656063 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c65f019-1f72-4024-a25f-c3f91e761b46 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:19.656360 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c65f019-1f72-4024-a25f-c3f91e761b46 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:19.678402 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c65f019-1f72-4024-a25f-c3f91e761b46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:14:19.688072 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-9c65f019-1f72-4024-a25f-c3f91e761b46 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 19:14:19.688072 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:14:19.688072 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:14:19.688072 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:14:19.688072 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:14:19.688072 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:14:19.688072 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:14:19.688072 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:14:19.688072 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:14:19.688072 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:14:19.688072 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:14:19.688072 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:14:19.688072 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:14:19.688072 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:14:19.688072 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:14:19.688072 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:14:19.688943 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:14:19.688943 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:14:19.688943 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:14:19.688943 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:14:19.688943 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:14:19.688943 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:14:19.688943 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:14:19.688943 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:14:19.688943 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:14:19.688943 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:14:19.688943 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:14:19.688943 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:14:19.688943 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:14:19.688943 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:14:19.688943 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:14:19.688943 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:14:19.689915 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:14:19.689915 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:14:19.689915 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:14:19.689915 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:14:19.689915 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:14:19.689915 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:14:19.689915 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:14:19.689915 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:14:19.689915 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:14:19.689915 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:14:19.689915 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 19:14:19.689915 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:14:19.689915 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 56/109] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 15 19:14:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 367 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:14:19.696727 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c65f019-1f72-4024-a25f-c3f91e761b46 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:19.724079 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4f4c3a-1143-4179-bda4-201b2dfee72b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:19.725574 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4f4c3a-1143-4179-bda4-201b2dfee72b None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:19.785844 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:19.875139 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 54/110] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 15 19:14:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 621 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 19:14:19.876558 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6b1a0b-f0b3-46a2-a55b-8d5749de17c1 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:19.876852 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6b1a0b-f0b3-46a2-a55b-8d5749de17c1 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:19.877134 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6b1a0b-f0b3-46a2-a55b-8d5749de17c1 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:19.878687 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 55/111] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:14:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:14:19.881298 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c65f019-1f72-4024-a25f-c3f91e761b46 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:19.884033 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad19444f-42e2-4e4b-b80b-25e04f280af5 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:19.884375 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad19444f-42e2-4e4b-b80b-25e04f280af5 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:19.884672 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad19444f-42e2-4e4b-b80b-25e04f280af5 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:19.886684 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 56/112] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:14:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:14:19.887449 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad19444f-42e2-4e4b-b80b-25e04f280af5 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:19.924608 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abbf65f-3bfb-4600-93e6-3a9b84c1eccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:19.925323 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abbf65f-3bfb-4600-93e6-3a9b84c1eccb None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:19.925611 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abbf65f-3bfb-4600-93e6-3a9b84c1eccb None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:19.925929 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abbf65f-3bfb-4600-93e6-3a9b84c1eccb None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:19.926693 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abbf65f-3bfb-4600-93e6-3a9b84c1eccb None admin] RBAC: Adding query filter params (name=member) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:19.927009 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abbf65f-3bfb-4600-93e6-3a9b84c1eccb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:19.928659 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abbf65f-3bfb-4600-93e6-3a9b84c1eccb None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:19.938407 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 57/113] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jul 15 19:14:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:14:19.939768 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc95171-31b4-4c05-99e2-b99ab3cea55a None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:19.940162 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc95171-31b4-4c05-99e2-b99ab3cea55a None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:19.940449 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc95171-31b4-4c05-99e2-b99ab3cea55a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:19.958247 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-9dc95171-31b4-4c05-99e2-b99ab3cea55a None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:14:19.959581 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9dc95171-31b4-4c05-99e2-b99ab3cea55a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:14:19.986157 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:19.999160 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad19444f-42e2-4e4b-b80b-25e04f280af5 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:20.029890 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:20.033711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 58/114] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:14:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:14:20.035034 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6accad26-6625-44c6-a404-0013b1a3f623 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:20.035334 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6accad26-6625-44c6-a404-0013b1a3f623 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:20.035602 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6accad26-6625-44c6-a404-0013b1a3f623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:20.048030 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e1db43-12c2-48e4-89cd-6337f8c81263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:20.048030 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e1db43-12c2-48e4-89cd-6337f8c81263 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:20.048030 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e1db43-12c2-48e4-89cd-6337f8c81263 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:20.048030 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e1db43-12c2-48e4-89cd-6337f8c81263 None admin] PATH_INFO: `/v3/projects/513641c037c84c54b779b5ca0684f17c` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:20.048394 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e1db43-12c2-48e4-89cd-6337f8c81263 None admin] RBAC: Authorizing `identity:get_project(project_id=513641c037c84c54b779b5ca0684f17c)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:20.052961 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e1db43-12c2-48e4-89cd-6337f8c81263 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:20.054853 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 57/115] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 15 19:14:19 2022] GET /identity/v3/projects/513641c037c84c54b779b5ca0684f17c => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:20.055715 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e1db43-12c2-48e4-89cd-6337f8c81263 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:20.063970 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-6accad26-6625-44c6-a404-0013b1a3f623 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:14:20.065168 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-6accad26-6625-44c6-a404-0013b1a3f623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:14:20.137165 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 59/116] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:14:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:14:20.146065 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6accad26-6625-44c6-a404-0013b1a3f623 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:20.165672 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e1db43-12c2-48e4-89cd-6337f8c81263 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:20.200644 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35928b7-44bc-4a31-b93d-71b820489352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:20.201560 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35928b7-44bc-4a31-b93d-71b820489352 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:20.201940 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35928b7-44bc-4a31-b93d-71b820489352 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:20.202224 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35928b7-44bc-4a31-b93d-71b820489352 None admin] PATH_INFO: `/v3/users/da619d0bd9ad4a48a65b481207b21979` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:20.203454 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35928b7-44bc-4a31-b93d-71b820489352 None admin] RBAC: Authorizing `identity:get_user(user_id=da619d0bd9ad4a48a65b481207b21979)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:20.204961 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35928b7-44bc-4a31-b93d-71b820489352 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:20.206662 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 58/117] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 15 19:14:20 2022] GET /identity/v3/users/da619d0bd9ad4a48a65b481207b21979 => generated 275 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:20.207434 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35928b7-44bc-4a31-b93d-71b820489352 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:20.237665 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:20.272536 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6accad26-6625-44c6-a404-0013b1a3f623 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:20.283926 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:20.310292 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca72f3c9-8377-4e28-a172-77438b4a4dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:20.311117 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca72f3c9-8377-4e28-a172-77438b4a4dc0 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:20.311407 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca72f3c9-8377-4e28-a172-77438b4a4dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:20.311690 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca72f3c9-8377-4e28-a172-77438b4a4dc0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:20.319660 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35928b7-44bc-4a31-b93d-71b820489352 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:20.322096 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca72f3c9-8377-4e28-a172-77438b4a4dc0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:14:20.330073 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-ca72f3c9-8377-4e28-a172-77438b4a4dc0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 19:14:20.330073 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:14:20.330073 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:14:20.330073 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:14:20.330073 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:14:20.330073 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:14:20.330073 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:14:20.330073 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:14:20.330073 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:14:20.330073 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:14:20.330073 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:14:20.330073 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:14:20.330073 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:14:20.330073 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:14:20.330073 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:14:20.330073 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:14:20.331055 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:14:20.331055 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:14:20.331055 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:14:20.331055 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:14:20.331055 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:14:20.331055 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:14:20.331055 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:14:20.331055 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:14:20.331055 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:14:20.331055 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:14:20.331055 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:14:20.331055 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:14:20.331055 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:14:20.331055 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:14:20.331055 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:14:20.331055 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:14:20.331815 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:14:20.331815 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:14:20.331815 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:14:20.331815 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:14:20.331815 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:14:20.331815 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:14:20.331815 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:14:20.331815 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:14:20.331815 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:14:20.331815 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:14:20.331815 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 19:14:20.331815 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:14:20.331815 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 60/118] 10.222.0.33 () {56 vars in 1114 bytes} [Fri Jul 15 19:14:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:14:20.332469 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca72f3c9-8377-4e28-a172-77438b4a4dc0 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:20.364277 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79c4922-5a4c-4b06-9419-df3d3bcdf55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:20.365205 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c4922-5a4c-4b06-9419-df3d3bcdf55f None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:20.365639 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c4922-5a4c-4b06-9419-df3d3bcdf55f None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:20.366094 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c4922-5a4c-4b06-9419-df3d3bcdf55f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:20.366939 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79c4922-5a4c-4b06-9419-df3d3bcdf55f None admin] RBAC: Adding query filter params (role.id=2f62da1ebb2b443eabf21e910a215048, scope.project.id=513641c037c84c54b779b5ca0684f17c, user.id=0505801f23f1420eb449bad3d0b36bec) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:20.367258 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79c4922-5a4c-4b06-9419-df3d3bcdf55f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:20.368827 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79c4922-5a4c-4b06-9419-df3d3bcdf55f None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:20.376072 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 59/119] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 15 19:14:20 2022] GET /identity/v3/role_assignments?user.id=0505801f23f1420eb449bad3d0b36bec&scope.project.id=513641c037c84c54b779b5ca0684f17c&role.id=2f62da1ebb2b443eabf21e910a215048 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:20.376956 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79c4922-5a4c-4b06-9419-df3d3bcdf55f None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:20.487440 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca72f3c9-8377-4e28-a172-77438b4a4dc0 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:20.498243 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79c4922-5a4c-4b06-9419-df3d3bcdf55f None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:20.535496 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:20.538122 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:20.539736 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3a05ab-b363-4d2b-8bf8-fdd6bd362e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:20.540426 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3a05ab-b363-4d2b-8bf8-fdd6bd362e56 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:20.540708 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3a05ab-b363-4d2b-8bf8-fdd6bd362e56 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:20.540987 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3a05ab-b363-4d2b-8bf8-fdd6bd362e56 None admin] PATH_INFO: `/v3/projects/513641c037c84c54b779b5ca0684f17c` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:20.542308 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3a05ab-b363-4d2b-8bf8-fdd6bd362e56 None admin] RBAC: Authorizing `identity:get_project(project_id=513641c037c84c54b779b5ca0684f17c)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:20.543773 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3a05ab-b363-4d2b-8bf8-fdd6bd362e56 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:20.544740 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adf663a-0cdb-40b9-b51d-ade65a8a4784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:20.545537 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adf663a-0cdb-40b9-b51d-ade65a8a4784 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:20.546012 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adf663a-0cdb-40b9-b51d-ade65a8a4784 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:20.550921 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adf663a-0cdb-40b9-b51d-ade65a8a4784 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:20.551020 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 60/120] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 15 19:14:20 2022] GET /identity/v3/projects/513641c037c84c54b779b5ca0684f17c => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:20.552699 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa598569-4c81-4782-afdf-203413f22485 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:20.552996 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa598569-4c81-4782-afdf-203413f22485 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:20.553284 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa598569-4c81-4782-afdf-203413f22485 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:20.555445 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 61/121] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:14:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:14:20.555952 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa598569-4c81-4782-afdf-203413f22485 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:20.560824 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adf663a-0cdb-40b9-b51d-ade65a8a4784 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:14:20.569897 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-3adf663a-0cdb-40b9-b51d-ade65a8a4784 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:14:20.569897 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:14:20.569897 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:14:20.569897 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:14:20.569897 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:14:20.569897 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:14:20.569897 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:14:20.569897 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:14:20.569897 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:14:20.569897 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:14:20.569897 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:14:20.569897 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:14:20.569897 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:14:20.569897 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:14:20.569897 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:14:20.569897 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:14:20.571685 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:14:20.571685 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:14:20.571685 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:14:20.571685 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:14:20.571685 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:14:20.571685 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:14:20.571685 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:14:20.571685 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:14:20.571685 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:14:20.571685 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:14:20.571685 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:14:20.571685 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:14:20.571685 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:14:20.571685 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:14:20.571685 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:14:20.571685 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:14:20.572478 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:14:20.572478 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:14:20.572478 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:14:20.572478 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:14:20.572478 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:14:20.572478 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:14:20.572478 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:14:20.572478 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:14:20.572478 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:14:20.572478 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:14:20.572478 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:14:20.572478 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:14:20.574215 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 61/122] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jul 15 19:14:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:14:20.574354 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adf663a-0cdb-40b9-b51d-ade65a8a4784 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:20.676098 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa598569-4c81-4782-afdf-203413f22485 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:20.712824 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e6c76b-3daa-40dd-a44f-6210e0fe01ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:20.713504 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e6c76b-3daa-40dd-a44f-6210e0fe01ac None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:20.713812 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e6c76b-3daa-40dd-a44f-6210e0fe01ac None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:20.713974 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adf663a-0cdb-40b9-b51d-ade65a8a4784 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:20.714140 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e6c76b-3daa-40dd-a44f-6210e0fe01ac None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:20.714845 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e6c76b-3daa-40dd-a44f-6210e0fe01ac None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:20.715157 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e6c76b-3daa-40dd-a44f-6210e0fe01ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:20.716601 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e6c76b-3daa-40dd-a44f-6210e0fe01ac None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:20.726431 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 62/123] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 15 19:14:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:20.727849 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1085da55-1813-4b98-a9d1-0e506cfe9903 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:20.728147 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1085da55-1813-4b98-a9d1-0e506cfe9903 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:20.728483 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1085da55-1813-4b98-a9d1-0e506cfe9903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:20.753711 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-1085da55-1813-4b98-a9d1-0e506cfe9903 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:14:20.755121 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-1085da55-1813-4b98-a9d1-0e506cfe9903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:14:20.755289 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b48e9e-525b-4575-b6f0-029c6ba15cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:20.756246 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b48e9e-525b-4575-b6f0-029c6ba15cfa None admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:20.756600 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b48e9e-525b-4575-b6f0-029c6ba15cfa None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:20.756949 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b48e9e-525b-4575-b6f0-029c6ba15cfa None admin] PATH_INFO: `/v3/projects/513641c037c84c54b779b5ca0684f17c/users/da619d0bd9ad4a48a65b481207b21979/roles/28763c6e70094116b3a4b2895895fa58` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:20.760919 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b48e9e-525b-4575-b6f0-029c6ba15cfa None admin] RBAC: Authorizing `identity:create_grant(project_id=513641c037c84c54b779b5ca0684f17c, user_id=da619d0bd9ad4a48a65b481207b21979, role_id=28763c6e70094116b3a4b2895895fa58)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:20.762512 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b48e9e-525b-4575-b6f0-029c6ba15cfa None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:20.819750 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:20.831315 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 63/124] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:14:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:14:20.838547 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1085da55-1813-4b98-a9d1-0e506cfe9903 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:20.865058 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 62/125] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jul 15 19:14:20 2022] PUT /identity/v3/projects/513641c037c84c54b779b5ca0684f17c/users/da619d0bd9ad4a48a65b481207b21979/roles/28763c6e70094116b3a4b2895895fa58 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:14:20.866494 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b48e9e-525b-4575-b6f0-029c6ba15cfa None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:20.984958 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:21.028719 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1085da55-1813-4b98-a9d1-0e506cfe9903 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:21.070813 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a72b18-1814-44a0-a206-21b3d1fa8e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:21.072463 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a72b18-1814-44a0-a206-21b3d1fa8e73 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:21.072981 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a72b18-1814-44a0-a206-21b3d1fa8e73 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:21.073451 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a72b18-1814-44a0-a206-21b3d1fa8e73 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:21.074435 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a72b18-1814-44a0-a206-21b3d1fa8e73 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:21.074956 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a72b18-1814-44a0-a206-21b3d1fa8e73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:21.076539 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a72b18-1814-44a0-a206-21b3d1fa8e73 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:21.084106 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b48e9e-525b-4575-b6f0-029c6ba15cfa None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:21.086015 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 64/126] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 15 19:14:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:21.087278 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a72b18-1814-44a0-a206-21b3d1fa8e73 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:21.130630 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d895127-b32e-4269-8cbb-ccd1dae0264f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:21.131576 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d895127-b32e-4269-8cbb-ccd1dae0264f None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:21.131857 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d895127-b32e-4269-8cbb-ccd1dae0264f None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:21.132134 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d895127-b32e-4269-8cbb-ccd1dae0264f None admin] PATH_INFO: `/v3/users/da619d0bd9ad4a48a65b481207b21979` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:21.133451 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d895127-b32e-4269-8cbb-ccd1dae0264f None admin] RBAC: Authorizing `identity:get_user(user_id=da619d0bd9ad4a48a65b481207b21979)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:21.135837 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d895127-b32e-4269-8cbb-ccd1dae0264f None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:21.141861 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 63/127] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 15 19:14:20 2022] GET /identity/v3/users/da619d0bd9ad4a48a65b481207b21979 => generated 275 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:21.142938 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d895127-b32e-4269-8cbb-ccd1dae0264f None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:21.153058 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:21.199011 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a72b18-1814-44a0-a206-21b3d1fa8e73 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:21.234370 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ec449a-b823-4035-bf58-9d431c75f88f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:21.235957 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ec449a-b823-4035-bf58-9d431c75f88f None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:21.236471 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ec449a-b823-4035-bf58-9d431c75f88f None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:21.236947 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ec449a-b823-4035-bf58-9d431c75f88f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:21.248516 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ec449a-b823-4035-bf58-9d431c75f88f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:14:21.256590 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-b1ec449a-b823-4035-bf58-9d431c75f88f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 19:14:21.256590 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:14:21.256590 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:14:21.256590 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:14:21.256590 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:14:21.256590 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:14:21.256590 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:14:21.256590 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:14:21.256590 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:14:21.256590 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:14:21.256590 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:14:21.256590 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:14:21.256590 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:14:21.256590 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:14:21.256590 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:14:21.256590 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:14:21.257446 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:14:21.257446 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:14:21.257446 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:14:21.257446 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:14:21.257446 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:14:21.257446 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:14:21.257446 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:14:21.257446 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:14:21.257446 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:14:21.257446 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:14:21.257446 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:14:21.257446 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:14:21.257446 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:14:21.257446 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:14:21.257446 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:14:21.257446 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:14:21.258269 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:14:21.258269 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:14:21.258269 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:14:21.258269 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:14:21.258269 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:14:21.258269 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:14:21.258269 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:14:21.258269 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:14:21.258269 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:14:21.258269 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:14:21.258269 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 19:14:21.258269 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:14:21.261062 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 65/128] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 15 19:14:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:14:21.262154 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ec449a-b823-4035-bf58-9d431c75f88f None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:21.349260 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d895127-b32e-4269-8cbb-ccd1dae0264f None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:21.383275 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:21.411239 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4df8dce-1acb-4fbb-9c64-6beb0ebe744d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:21.413216 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4df8dce-1acb-4fbb-9c64-6beb0ebe744d None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:21.414383 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4df8dce-1acb-4fbb-9c64-6beb0ebe744d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:21.415555 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4df8dce-1acb-4fbb-9c64-6beb0ebe744d None admin] PATH_INFO: `/v3/users/96f47b71ea5a45ad92368f6b4130c36c` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:21.419032 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4df8dce-1acb-4fbb-9c64-6beb0ebe744d None admin] RBAC: Authorizing `identity:get_user(user_id=96f47b71ea5a45ad92368f6b4130c36c)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:21.421354 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4df8dce-1acb-4fbb-9c64-6beb0ebe744d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:21.425572 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 64/129] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 15 19:14:21 2022] GET /identity/v3/users/96f47b71ea5a45ad92368f6b4130c36c => generated 247 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:21.427202 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4df8dce-1acb-4fbb-9c64-6beb0ebe744d None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:21.429755 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ec449a-b823-4035-bf58-9d431c75f88f None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:21.455727 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:21.489182 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38672f0-78b5-4c6b-a912-00b6fd2736e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:21.492282 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38672f0-78b5-4c6b-a912-00b6fd2736e6 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:21.493923 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38672f0-78b5-4c6b-a912-00b6fd2736e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:21.494574 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38672f0-78b5-4c6b-a912-00b6fd2736e6 None admin] PATH_INFO: `/v3/projects/513641c037c84c54b779b5ca0684f17c` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:21.498722 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38672f0-78b5-4c6b-a912-00b6fd2736e6 None admin] RBAC: Authorizing `identity:get_project(project_id=513641c037c84c54b779b5ca0684f17c)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:21.501222 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38672f0-78b5-4c6b-a912-00b6fd2736e6 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:21.506723 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 66/130] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 15 19:14:21 2022] GET /identity/v3/projects/513641c037c84c54b779b5ca0684f17c => generated 298 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:21.507765 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38672f0-78b5-4c6b-a912-00b6fd2736e6 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:21.586785 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4df8dce-1acb-4fbb-9c64-6beb0ebe744d None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:21.616247 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38672f0-78b5-4c6b-a912-00b6fd2736e6 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:21.628967 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb968844-ccc6-4dc2-854a-649f42149c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:21.630001 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb968844-ccc6-4dc2-854a-649f42149c55 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:21.630285 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb968844-ccc6-4dc2-854a-649f42149c55 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:21.630563 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb968844-ccc6-4dc2-854a-649f42149c55 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:21.631321 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb968844-ccc6-4dc2-854a-649f42149c55 None admin] RBAC: Adding query filter params (name=member) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:21.631626 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb968844-ccc6-4dc2-854a-649f42149c55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:21.633531 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb968844-ccc6-4dc2-854a-649f42149c55 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:21.645155 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 65/131] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jul 15 19:14:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:21.645884 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb968844-ccc6-4dc2-854a-649f42149c55 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:21.661431 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d87e83e-219c-4a91-b5cf-f206619fa1a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:21.662793 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d87e83e-219c-4a91-b5cf-f206619fa1a7 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:21.663281 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d87e83e-219c-4a91-b5cf-f206619fa1a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:21.663754 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d87e83e-219c-4a91-b5cf-f206619fa1a7 None admin] PATH_INFO: `/v3/projects/513641c037c84c54b779b5ca0684f17c` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:21.665244 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d87e83e-219c-4a91-b5cf-f206619fa1a7 None admin] RBAC: Authorizing `identity:get_project(project_id=513641c037c84c54b779b5ca0684f17c)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:21.666934 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d87e83e-219c-4a91-b5cf-f206619fa1a7 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:21.668785 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 67/132] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 15 19:14:21 2022] GET /identity/v3/projects/513641c037c84c54b779b5ca0684f17c => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:21.670124 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6083024-0783-4c49-9019-695db8df418a None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:21.670512 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6083024-0783-4c49-9019-695db8df418a None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:21.670889 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6083024-0783-4c49-9019-695db8df418a None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:21.672236 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 68/133] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:14:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:14:21.676462 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6083024-0783-4c49-9019-695db8df418a None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:21.725387 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:21.729661 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:21.791804 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6083024-0783-4c49-9019-695db8df418a None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:21.795300 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb968844-ccc6-4dc2-854a-649f42149c55 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:21.833235 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf24a65-018d-4096-b80b-af83f14a9964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:21.834295 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf24a65-018d-4096-b80b-af83f14a9964 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:21.834685 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf24a65-018d-4096-b80b-af83f14a9964 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:21.835074 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf24a65-018d-4096-b80b-af83f14a9964 None admin] PATH_INFO: `/v3/users/da619d0bd9ad4a48a65b481207b21979` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:21.836522 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf24a65-018d-4096-b80b-af83f14a9964 None admin] RBAC: Authorizing `identity:get_user(user_id=da619d0bd9ad4a48a65b481207b21979)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:21.838038 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7ab727-cc93-4fb7-a1cf-467f4a5eb481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:21.838745 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf24a65-018d-4096-b80b-af83f14a9964 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:21.839304 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7ab727-cc93-4fb7-a1cf-467f4a5eb481 None admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:21.839727 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7ab727-cc93-4fb7-a1cf-467f4a5eb481 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:21.840164 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7ab727-cc93-4fb7-a1cf-467f4a5eb481 None admin] PATH_INFO: `/v3/projects/513641c037c84c54b779b5ca0684f17c/users/da619d0bd9ad4a48a65b481207b21979/roles/e97d26621445466d9550261c608db9c5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:21.845237 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 69/134] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 15 19:14:21 2022] GET /identity/v3/users/da619d0bd9ad4a48a65b481207b21979 => generated 275 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:21.845237 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf24a65-018d-4096-b80b-af83f14a9964 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:21.847967 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7ab727-cc93-4fb7-a1cf-467f4a5eb481 None admin] RBAC: Authorizing `identity:create_grant(project_id=513641c037c84c54b779b5ca0684f17c, user_id=da619d0bd9ad4a48a65b481207b21979, role_id=e97d26621445466d9550261c608db9c5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:21.850708 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7ab727-cc93-4fb7-a1cf-467f4a5eb481 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:21.955637 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf24a65-018d-4096-b80b-af83f14a9964 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:21.972966 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:21.989022 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 66/135] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jul 15 19:14:21 2022] PUT /identity/v3/projects/513641c037c84c54b779b5ca0684f17c/users/da619d0bd9ad4a48a65b481207b21979/roles/e97d26621445466d9550261c608db9c5 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:14:21.991714 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f48c8e-6002-46d9-b329-1ce2e0574827 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:21.992234 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f48c8e-6002-46d9-b329-1ce2e0574827 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:21.992744 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f48c8e-6002-46d9-b329-1ce2e0574827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:21.995731 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49f84a7-fa20-439a-8597-15f445ceea3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:21.996707 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49f84a7-fa20-439a-8597-15f445ceea3d None admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:21.996995 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49f84a7-fa20-439a-8597-15f445ceea3d None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:21.997275 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49f84a7-fa20-439a-8597-15f445ceea3d None admin] PATH_INFO: `/v3/projects/513641c037c84c54b779b5ca0684f17c/users/96f47b71ea5a45ad92368f6b4130c36c/roles/4318f86ba9f74cd4a0bef1c311276657` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:21.999557 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49f84a7-fa20-439a-8597-15f445ceea3d None admin] RBAC: Authorizing `identity:create_grant(project_id=513641c037c84c54b779b5ca0684f17c, user_id=96f47b71ea5a45ad92368f6b4130c36c, role_id=4318f86ba9f74cd4a0bef1c311276657)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:22.001251 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49f84a7-fa20-439a-8597-15f445ceea3d None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:22.016083 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-a4f48c8e-6002-46d9-b329-1ce2e0574827 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:14:22.017802 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-a4f48c8e-6002-46d9-b329-1ce2e0574827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:14:22.090739 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 70/136] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jul 15 19:14:21 2022] PUT /identity/v3/projects/513641c037c84c54b779b5ca0684f17c/users/96f47b71ea5a45ad92368f6b4130c36c/roles/4318f86ba9f74cd4a0bef1c311276657 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:14:22.091895 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49f84a7-fa20-439a-8597-15f445ceea3d None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:22.101745 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 67/137] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:14:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:14:22.128935 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f48c8e-6002-46d9-b329-1ce2e0574827 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:22.175906 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:22.256956 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f48c8e-6002-46d9-b329-1ce2e0574827 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:22.293348 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49f84a7-fa20-439a-8597-15f445ceea3d None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:22.299107 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77067a8-0a98-4951-b72c-c3e6f9d48660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:22.300240 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77067a8-0a98-4951-b72c-c3e6f9d48660 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:22.300724 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77067a8-0a98-4951-b72c-c3e6f9d48660 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:22.301290 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77067a8-0a98-4951-b72c-c3e6f9d48660 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:22.302763 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77067a8-0a98-4951-b72c-c3e6f9d48660 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:22.304619 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77067a8-0a98-4951-b72c-c3e6f9d48660 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:22.307161 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 68/138] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:14:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:22.313255 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77067a8-0a98-4951-b72c-c3e6f9d48660 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:22.331441 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffd0fac-2fcb-48f8-a38f-daa6a4e201e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:22.332345 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffd0fac-2fcb-48f8-a38f-daa6a4e201e5 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:22.332910 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffd0fac-2fcb-48f8-a38f-daa6a4e201e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:22.333016 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffd0fac-2fcb-48f8-a38f-daa6a4e201e5 None admin] PATH_INFO: `/v3/projects/e733f93067e74ac193989bcd00d8d5ce` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:22.334265 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffd0fac-2fcb-48f8-a38f-daa6a4e201e5 None admin] RBAC: Authorizing `identity:get_project(project_id=e733f93067e74ac193989bcd00d8d5ce)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:22.336100 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffd0fac-2fcb-48f8-a38f-daa6a4e201e5 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:22.338935 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 71/139] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 15 19:14:22 2022] GET /identity/v3/projects/e733f93067e74ac193989bcd00d8d5ce => generated 312 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:22.346883 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffd0fac-2fcb-48f8-a38f-daa6a4e201e5 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:22.400874 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:22.435799 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77067a8-0a98-4951-b72c-c3e6f9d48660 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:22.444635 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:22.461134 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffd0fac-2fcb-48f8-a38f-daa6a4e201e5 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:22.475127 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1239424e-880d-4023-a670-ee2fe9f2f7db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:22.475972 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1239424e-880d-4023-a670-ee2fe9f2f7db None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:22.476343 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1239424e-880d-4023-a670-ee2fe9f2f7db None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:22.476726 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1239424e-880d-4023-a670-ee2fe9f2f7db None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:22.478827 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1239424e-880d-4023-a670-ee2fe9f2f7db None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:22.480455 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1239424e-880d-4023-a670-ee2fe9f2f7db None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:22.503057 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb79e95-4757-437d-8015-c73e6b6baafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:22.504215 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb79e95-4757-437d-8015-c73e6b6baafe None admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:22.504646 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb79e95-4757-437d-8015-c73e6b6baafe None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:22.505112 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb79e95-4757-437d-8015-c73e6b6baafe None admin] PATH_INFO: `/v3/projects/e733f93067e74ac193989bcd00d8d5ce/users/da619d0bd9ad4a48a65b481207b21979/roles/28763c6e70094116b3a4b2895895fa58` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:22.508336 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb79e95-4757-437d-8015-c73e6b6baafe None admin] RBAC: Authorizing `identity:create_grant(project_id=e733f93067e74ac193989bcd00d8d5ce, user_id=da619d0bd9ad4a48a65b481207b21979, role_id=28763c6e70094116b3a4b2895895fa58)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:22.510028 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb79e95-4757-437d-8015-c73e6b6baafe None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:22.600582 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 69/140] 10.222.0.33 () {60 vars in 1132 bytes} [Fri Jul 15 19:14:22 2022] POST /identity/v3/users => generated 283 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:14:22.630931 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 72/141] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jul 15 19:14:22 2022] PUT /identity/v3/projects/e733f93067e74ac193989bcd00d8d5ce/users/da619d0bd9ad4a48a65b481207b21979/roles/28763c6e70094116b3a4b2895895fa58 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:14:22.854891 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f890051-8f28-476c-926a-34d75e3b9b49 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:22.855214 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f890051-8f28-476c-926a-34d75e3b9b49 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:22.855457 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f890051-8f28-476c-926a-34d75e3b9b49 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:22.857010 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 70/142] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:14:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:14:22.871673 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e59c02-3c10-4f22-92bf-091c28aa7eb6 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:22.872404 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e59c02-3c10-4f22-92bf-091c28aa7eb6 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:22.872885 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e59c02-3c10-4f22-92bf-091c28aa7eb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:22.894920 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-f5e59c02-3c10-4f22-92bf-091c28aa7eb6 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:14:22.896562 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-f5e59c02-3c10-4f22-92bf-091c28aa7eb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:14:22.978026 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 73/143] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:14:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:14:23.003047 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f890051-8f28-476c-926a-34d75e3b9b49 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:23.132482 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f890051-8f28-476c-926a-34d75e3b9b49 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:23.174312 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aead28a-b09b-46a6-b1cf-17dcc335cc01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:23.175485 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aead28a-b09b-46a6-b1cf-17dcc335cc01 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:23.176257 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aead28a-b09b-46a6-b1cf-17dcc335cc01 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:23.177174 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aead28a-b09b-46a6-b1cf-17dcc335cc01 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:23.189303 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aead28a-b09b-46a6-b1cf-17dcc335cc01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:14:23.203436 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-4aead28a-b09b-46a6-b1cf-17dcc335cc01 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 19:14:23.203436 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:14:23.203436 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:14:23.203436 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:14:23.203436 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:14:23.203436 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:14:23.203436 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:14:23.203436 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:14:23.203436 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:14:23.203436 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:14:23.203436 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:14:23.203436 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:14:23.203436 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:14:23.203436 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:14:23.203436 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:14:23.203436 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:14:23.204373 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:14:23.204373 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:14:23.204373 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:14:23.204373 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:14:23.204373 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:14:23.204373 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:14:23.204373 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:14:23.204373 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:14:23.204373 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:14:23.204373 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:14:23.204373 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:14:23.204373 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:14:23.204373 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:14:23.204373 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:14:23.204373 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:14:23.204373 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:14:23.205134 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:14:23.205134 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:14:23.205134 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:14:23.205134 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:14:23.205134 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:14:23.205134 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:14:23.205134 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:14:23.205134 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:14:23.205134 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:14:23.205134 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:14:23.205134 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 19:14:23.205134 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:14:23.207596 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 71/144] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 15 19:14:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:14:23.213926 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e59c02-3c10-4f22-92bf-091c28aa7eb6 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:23.275270 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:23.366709 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e59c02-3c10-4f22-92bf-091c28aa7eb6 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:23.385818 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3d08fb-480a-47cc-9deb-48d438783225 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:23.386440 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3d08fb-480a-47cc-9deb-48d438783225 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:23.386954 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3d08fb-480a-47cc-9deb-48d438783225 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:23.392619 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 72/145] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:14:23 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:14:23.394497 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfcc078-b2fe-464b-ba66-4ca37c640ec2 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:23.395004 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfcc078-b2fe-464b-ba66-4ca37c640ec2 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:23.395482 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfcc078-b2fe-464b-ba66-4ca37c640ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:23.410491 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30de2770-a6bc-4036-ac74-4333f5508eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:23.411262 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30de2770-a6bc-4036-ac74-4333f5508eed None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:23.411457 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30de2770-a6bc-4036-ac74-4333f5508eed None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:23.411732 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30de2770-a6bc-4036-ac74-4333f5508eed None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:23.412490 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30de2770-a6bc-4036-ac74-4333f5508eed None admin] RBAC: Adding query filter params (name=reader) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:23.412790 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30de2770-a6bc-4036-ac74-4333f5508eed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:23.414374 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30de2770-a6bc-4036-ac74-4333f5508eed None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:23.417937 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-ccfcc078-b2fe-464b-ba66-4ca37c640ec2 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:14:23.419450 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-ccfcc078-b2fe-464b-ba66-4ca37c640ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:14:23.424483 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 74/146] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jul 15 19:14:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:23.430232 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30de2770-a6bc-4036-ac74-4333f5508eed None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:23.443730 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:23.548606 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 73/147] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:14:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 19:14:23.580080 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:23.582765 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfcc078-b2fe-464b-ba66-4ca37c640ec2 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:23.584494 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30de2770-a6bc-4036-ac74-4333f5508eed None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:23.642034 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ce4a55-3d3c-4d25-a8b3-783ff33be88f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:23.642837 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ce4a55-3d3c-4d25-a8b3-783ff33be88f None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:23.643119 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ce4a55-3d3c-4d25-a8b3-783ff33be88f None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:23.643391 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ce4a55-3d3c-4d25-a8b3-783ff33be88f None admin] PATH_INFO: `/v3/users/0505801f23f1420eb449bad3d0b36bec` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:23.646565 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ce4a55-3d3c-4d25-a8b3-783ff33be88f None admin] RBAC: Authorizing `identity:get_user(user_id=0505801f23f1420eb449bad3d0b36bec)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:23.650252 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ce4a55-3d3c-4d25-a8b3-783ff33be88f None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:23.650374 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 75/148] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 15 19:14:23 2022] GET /identity/v3/users/0505801f23f1420eb449bad3d0b36bec => generated 289 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:14:23.661598 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ce4a55-3d3c-4d25-a8b3-783ff33be88f None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:23.764596 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfcc078-b2fe-464b-ba66-4ca37c640ec2 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:23.797806 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ce4a55-3d3c-4d25-a8b3-783ff33be88f None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:23.819435 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e3c245-dce5-4ee5-b45b-93577fa06861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:23.820589 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e3c245-dce5-4ee5-b45b-93577fa06861 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:23.821132 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e3c245-dce5-4ee5-b45b-93577fa06861 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:23.821626 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e3c245-dce5-4ee5-b45b-93577fa06861 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:23.844776 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a16f7b-0569-456d-a41d-c403f782f80d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:23.845331 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e3c245-dce5-4ee5-b45b-93577fa06861 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:14:23.845693 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a16f7b-0569-456d-a41d-c403f782f80d None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:23.846000 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a16f7b-0569-456d-a41d-c403f782f80d None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:23.846277 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a16f7b-0569-456d-a41d-c403f782f80d None admin] PATH_INFO: `/v3/projects/513641c037c84c54b779b5ca0684f17c` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:23.850512 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a16f7b-0569-456d-a41d-c403f782f80d None admin] RBAC: Authorizing `identity:get_project(project_id=513641c037c84c54b779b5ca0684f17c)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:23.852012 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a16f7b-0569-456d-a41d-c403f782f80d None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:23.857788 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 76/149] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 15 19:14:23 2022] GET /identity/v3/projects/513641c037c84c54b779b5ca0684f17c => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:14:23.859523 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-46e3c245-dce5-4ee5-b45b-93577fa06861 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 19:14:23.859523 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:14:23.859523 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:14:23.859523 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:14:23.859523 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:14:23.859523 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:14:23.859523 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:14:23.859523 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:14:23.859523 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:14:23.859523 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:14:23.859523 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:14:23.859523 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:14:23.859523 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:14:23.859523 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:14:23.859523 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:14:23.859523 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:14:23.860370 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:14:23.860370 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:14:23.860370 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:14:23.860370 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:14:23.860370 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:14:23.860370 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:14:23.860370 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:14:23.860370 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:14:23.860370 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:14:23.860370 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:14:23.860370 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:14:23.860370 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:14:23.860370 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:14:23.860370 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:14:23.860370 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:14:23.860370 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:14:23.861137 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:14:23.861137 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:14:23.861137 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:14:23.861137 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:14:23.861137 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:14:23.861137 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:14:23.861137 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:14:23.861137 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:14:23.861137 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:14:23.861137 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:14:23.861137 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 19:14:23.861137 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:14:23.863773 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 74/150] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 15 19:14:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:14:23.869824 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a16f7b-0569-456d-a41d-c403f782f80d None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:23.870978 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e3c245-dce5-4ee5-b45b-93577fa06861 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:23.894277 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:23.925219 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:24.012977 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a16f7b-0569-456d-a41d-c403f782f80d None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:24.054155 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a017cf4e-0879-4bd5-86c1-e5eeb67cbbff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:24.055094 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a017cf4e-0879-4bd5-86c1-e5eeb67cbbff None admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:24.055382 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a017cf4e-0879-4bd5-86c1-e5eeb67cbbff None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:24.056192 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a017cf4e-0879-4bd5-86c1-e5eeb67cbbff None admin] PATH_INFO: `/v3/projects/513641c037c84c54b779b5ca0684f17c/users/0505801f23f1420eb449bad3d0b36bec/roles/2f62da1ebb2b443eabf21e910a215048` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:24.061786 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a017cf4e-0879-4bd5-86c1-e5eeb67cbbff None admin] RBAC: Authorizing `identity:create_grant(project_id=513641c037c84c54b779b5ca0684f17c, user_id=0505801f23f1420eb449bad3d0b36bec, role_id=2f62da1ebb2b443eabf21e910a215048)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:24.061786 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a017cf4e-0879-4bd5-86c1-e5eeb67cbbff None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:24.110718 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e3c245-dce5-4ee5-b45b-93577fa06861 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:24.157068 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf4e820-bcea-4351-8e37-914acabcb2c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:24.158116 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf4e820-bcea-4351-8e37-914acabcb2c9 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:24.158590 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf4e820-bcea-4351-8e37-914acabcb2c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:24.159023 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf4e820-bcea-4351-8e37-914acabcb2c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:24.170365 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf4e820-bcea-4351-8e37-914acabcb2c9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:24.170365 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf4e820-bcea-4351-8e37-914acabcb2c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:24.170365 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf4e820-bcea-4351-8e37-914acabcb2c9 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:24.170612 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 77/151] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jul 15 19:14:23 2022] PUT /identity/v3/projects/513641c037c84c54b779b5ca0684f17c/users/0505801f23f1420eb449bad3d0b36bec/roles/2f62da1ebb2b443eabf21e910a215048 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:14:24.174100 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 75/152] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jul 15 19:14:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:24.181336 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a017cf4e-0879-4bd5-86c1-e5eeb67cbbff None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:24.303453 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:24.311120 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a017cf4e-0879-4bd5-86c1-e5eeb67cbbff None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:24.351799 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2cc086-2b99-4050-8ec4-904de5ae47f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:24.352744 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2cc086-2b99-4050-8ec4-904de5ae47f6 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:24.353029 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2cc086-2b99-4050-8ec4-904de5ae47f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:24.353308 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2cc086-2b99-4050-8ec4-904de5ae47f6 None admin] PATH_INFO: `/v3/users/da619d0bd9ad4a48a65b481207b21979` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:24.358878 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2cc086-2b99-4050-8ec4-904de5ae47f6 None admin] RBAC: Authorizing `identity:get_user(user_id=da619d0bd9ad4a48a65b481207b21979)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:24.360496 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2cc086-2b99-4050-8ec4-904de5ae47f6 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:24.362647 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 78/153] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 15 19:14:24 2022] GET /identity/v3/users/da619d0bd9ad4a48a65b481207b21979 => generated 275 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:24.368539 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf4e820-bcea-4351-8e37-914acabcb2c9 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:24.497178 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:24.538842 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf4e820-bcea-4351-8e37-914acabcb2c9 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:24.586699 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7998e93-c87f-4484-afec-f088ba565ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:24.587397 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7998e93-c87f-4484-afec-f088ba565ff3 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:24.587682 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7998e93-c87f-4484-afec-f088ba565ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:24.587957 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7998e93-c87f-4484-afec-f088ba565ff3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:24.667732 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7998e93-c87f-4484-afec-f088ba565ff3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:24.669221 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7998e93-c87f-4484-afec-f088ba565ff3 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:24.758271 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:24.792060 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 76/154] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 15 19:14:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 19:14:24.798114 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2cc086-2b99-4050-8ec4-904de5ae47f6 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:24.966765 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2cc086-2b99-4050-8ec4-904de5ae47f6 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:25.014778 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d97ac9-0283-4f73-ab2e-c2a5a6e60baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:25.015995 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d97ac9-0283-4f73-ab2e-c2a5a6e60baa None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:25.016492 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d97ac9-0283-4f73-ab2e-c2a5a6e60baa None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:25.017058 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d97ac9-0283-4f73-ab2e-c2a5a6e60baa None admin] PATH_INFO: `/v3/projects/513641c037c84c54b779b5ca0684f17c` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:25.018476 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d97ac9-0283-4f73-ab2e-c2a5a6e60baa None admin] RBAC: Authorizing `identity:get_project(project_id=513641c037c84c54b779b5ca0684f17c)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:25.020188 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d97ac9-0283-4f73-ab2e-c2a5a6e60baa None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:25.026222 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 79/155] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 15 19:14:24 2022] GET /identity/v3/projects/513641c037c84c54b779b5ca0684f17c => generated 298 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:25.032337 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7998e93-c87f-4484-afec-f088ba565ff3 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:25.173011 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7998e93-c87f-4484-afec-f088ba565ff3 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:25.217399 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be949ca3-1b4e-41f5-8f6f-06e6b39342b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:25.218096 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be949ca3-1b4e-41f5-8f6f-06e6b39342b1 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:25.218382 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be949ca3-1b4e-41f5-8f6f-06e6b39342b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:25.218661 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be949ca3-1b4e-41f5-8f6f-06e6b39342b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:25.219501 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be949ca3-1b4e-41f5-8f6f-06e6b39342b1 None admin] RBAC: Adding query filter params (role.id=28763c6e70094116b3a4b2895895fa58, scope.project.id=513641c037c84c54b779b5ca0684f17c, user.id=da619d0bd9ad4a48a65b481207b21979) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:25.219808 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be949ca3-1b4e-41f5-8f6f-06e6b39342b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:25.221258 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be949ca3-1b4e-41f5-8f6f-06e6b39342b1 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:25.227683 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 77/156] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 15 19:14:25 2022] GET /identity/v3/role_assignments?user.id=da619d0bd9ad4a48a65b481207b21979&scope.project.id=513641c037c84c54b779b5ca0684f17c&role.id=28763c6e70094116b3a4b2895895fa58 => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:25.686229 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4484a22-6d40-4976-8f51-9f6f18d8b97c None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:25.686923 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4484a22-6d40-4976-8f51-9f6f18d8b97c None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:25.687480 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4484a22-6d40-4976-8f51-9f6f18d8b97c None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:25.689314 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 80/157] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:14:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:14:25.694296 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c872c66-e8ba-42ee-addb-03a40bcb9902 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:25.695148 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c872c66-e8ba-42ee-addb-03a40bcb9902 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:25.695640 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c872c66-e8ba-42ee-addb-03a40bcb9902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:25.722768 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-8c872c66-e8ba-42ee-addb-03a40bcb9902 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:14:25.724334 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-8c872c66-e8ba-42ee-addb-03a40bcb9902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:14:25.766551 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:25.825448 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 78/158] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:14:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 19:14:25.867226 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4484a22-6d40-4976-8f51-9f6f18d8b97c None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:25.916099 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:25.986888 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b98c70-09c3-443b-980c-55d41d929649 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:25.987236 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b98c70-09c3-443b-980c-55d41d929649 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:25.987431 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b98c70-09c3-443b-980c-55d41d929649 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:25.988751 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 79/159] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:14:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:14:25.994596 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8659622-d92a-4f46-9a23-de4d3143c7b3 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:25.994872 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8659622-d92a-4f46-9a23-de4d3143c7b3 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:25.995142 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8659622-d92a-4f46-9a23-de4d3143c7b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:26.007342 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4484a22-6d40-4976-8f51-9f6f18d8b97c None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:26.013116 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-b8659622-d92a-4f46-9a23-de4d3143c7b3 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:14:26.014391 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b8659622-d92a-4f46-9a23-de4d3143c7b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:14:26.076632 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f9407d-1b6a-4adf-9cfe-b422f84adb82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:26.077566 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f9407d-1b6a-4adf-9cfe-b422f84adb82 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:26.077875 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f9407d-1b6a-4adf-9cfe-b422f84adb82 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:26.078153 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f9407d-1b6a-4adf-9cfe-b422f84adb82 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:26.103887 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 80/160] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:14:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 19:14:26.110541 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f9407d-1b6a-4adf-9cfe-b422f84adb82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:14:26.126834 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-20f9407d-1b6a-4adf-9cfe-b422f84adb82 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 19:14:26.126834 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:14:26.126834 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:14:26.126834 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:14:26.126834 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:14:26.126834 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:14:26.126834 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:14:26.126834 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:14:26.126834 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:14:26.126834 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:14:26.126834 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:14:26.126834 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:14:26.126834 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:14:26.126834 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:14:26.126834 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:14:26.126834 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:14:26.127715 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:14:26.127715 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:14:26.127715 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:14:26.127715 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:14:26.127715 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:14:26.127715 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:14:26.127715 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:14:26.127715 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:14:26.127715 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:14:26.127715 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:14:26.127715 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:14:26.127715 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:14:26.127715 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:14:26.127715 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:14:26.127715 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:14:26.127715 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:14:26.129585 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:14:26.129585 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:14:26.129585 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:14:26.129585 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:14:26.129585 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:14:26.129585 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:14:26.129585 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:14:26.129585 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:14:26.129585 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:14:26.129585 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:14:26.129585 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 19:14:26.129585 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:14:26.129585 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 81/161] 10.222.0.33 () {56 vars in 1114 bytes} [Fri Jul 15 19:14:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:14:26.137590 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8659622-d92a-4f46-9a23-de4d3143c7b3 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:26.148830 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7635e1-9be3-4524-be5d-7064d89a10b7 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:26.149412 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7635e1-9be3-4524-be5d-7064d89a10b7 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:26.149693 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7635e1-9be3-4524-be5d-7064d89a10b7 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:26.151189 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 82/162] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:14:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 19:14:26.152998 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7635e1-9be3-4524-be5d-7064d89a10b7 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:26.255702 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:26.266452 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7635e1-9be3-4524-be5d-7064d89a10b7 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:26.301978 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d97dd9-fb80-4065-9bba-576d1c0fd60b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:26.302784 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d97dd9-fb80-4065-9bba-576d1c0fd60b None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:26.303066 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d97dd9-fb80-4065-9bba-576d1c0fd60b None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:26.303341 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d97dd9-fb80-4065-9bba-576d1c0fd60b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:26.315140 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d97dd9-fb80-4065-9bba-576d1c0fd60b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:14:26.323772 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-37d97dd9-fb80-4065-9bba-576d1c0fd60b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 19:14:26.323772 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:14:26.323772 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:14:26.323772 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:14:26.323772 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:14:26.323772 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:14:26.323772 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:14:26.323772 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:14:26.323772 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:14:26.323772 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:14:26.323772 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:14:26.323772 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:14:26.323772 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:14:26.323772 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:14:26.323772 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:14:26.323772 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:14:26.324801 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:14:26.324801 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:14:26.324801 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:14:26.324801 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:14:26.324801 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:14:26.324801 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:14:26.324801 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:14:26.324801 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:14:26.324801 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:14:26.324801 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:14:26.324801 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:14:26.324801 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:14:26.324801 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:14:26.324801 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:14:26.324801 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:14:26.324801 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:14:26.326681 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:14:26.326681 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:14:26.326681 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:14:26.326681 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:14:26.326681 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:14:26.326681 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:14:26.326681 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:14:26.326681 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:14:26.326681 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:14:26.326681 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:14:26.326681 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 19:14:26.326681 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:14:26.326681 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 83/163] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 15 19:14:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:14:26.326681 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4253a9-b630-49ba-b4f7-0111168ec3f4 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:26.327362 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4253a9-b630-49ba-b4f7-0111168ec3f4 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:26.327362 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4253a9-b630-49ba-b4f7-0111168ec3f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:26.346189 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-fd4253a9-b630-49ba-b4f7-0111168ec3f4 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:14:26.347425 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-fd4253a9-b630-49ba-b4f7-0111168ec3f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:14:26.372306 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:26.379896 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8659622-d92a-4f46-9a23-de4d3143c7b3 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:26.420165 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 84/164] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:14:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:14:26.420851 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4253a9-b630-49ba-b4f7-0111168ec3f4 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:26.498494 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8b4cda-dc0d-47c5-886e-c864f4abe5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:26.499508 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8b4cda-dc0d-47c5-886e-c864f4abe5f7 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:26.499963 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8b4cda-dc0d-47c5-886e-c864f4abe5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:26.500407 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8b4cda-dc0d-47c5-886e-c864f4abe5f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:26.501368 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8b4cda-dc0d-47c5-886e-c864f4abe5f7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:26.501878 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8b4cda-dc0d-47c5-886e-c864f4abe5f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:26.503454 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8b4cda-dc0d-47c5-886e-c864f4abe5f7 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:26.517314 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 81/165] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 15 19:14:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 380 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:26.519485 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8b4cda-dc0d-47c5-886e-c864f4abe5f7 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:26.550629 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4253a9-b630-49ba-b4f7-0111168ec3f4 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:26.554079 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:26.586168 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51d35c4-b515-40fa-962f-6284805ea6ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:26.587432 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51d35c4-b515-40fa-962f-6284805ea6ab None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:26.587688 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51d35c4-b515-40fa-962f-6284805ea6ab None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:26.587961 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51d35c4-b515-40fa-962f-6284805ea6ab None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:26.588722 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51d35c4-b515-40fa-962f-6284805ea6ab None admin] RBAC: Adding query filter params (name=member) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:26.589273 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51d35c4-b515-40fa-962f-6284805ea6ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:26.590490 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51d35c4-b515-40fa-962f-6284805ea6ab None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:26.599927 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 85/166] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jul 15 19:14:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:26.600755 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51d35c4-b515-40fa-962f-6284805ea6ab None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:26.629032 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8b4cda-dc0d-47c5-886e-c864f4abe5f7 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:26.665804 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bee96f-7861-4bd0-8404-46ab74021e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:26.666998 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bee96f-7861-4bd0-8404-46ab74021e87 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:26.667493 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bee96f-7861-4bd0-8404-46ab74021e87 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:26.667975 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bee96f-7861-4bd0-8404-46ab74021e87 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:26.679159 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bee96f-7861-4bd0-8404-46ab74021e87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:14:26.687692 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-b5bee96f-7861-4bd0-8404-46ab74021e87 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:14:26.687692 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:14:26.687692 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:14:26.687692 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:14:26.687692 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:14:26.687692 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:14:26.687692 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:14:26.687692 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:14:26.687692 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:14:26.687692 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:14:26.687692 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:14:26.687692 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:14:26.687692 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:14:26.687692 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:14:26.687692 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:14:26.687692 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:14:26.688711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:14:26.688711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:14:26.688711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:14:26.688711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:14:26.688711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:14:26.688711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:14:26.688711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:14:26.688711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:14:26.688711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:14:26.688711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:14:26.688711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:14:26.688711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:14:26.688711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:14:26.688711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:14:26.688711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:14:26.688711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:14:26.689491 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:14:26.689491 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:14:26.689491 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:14:26.689491 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:14:26.689491 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:14:26.689491 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:14:26.689491 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:14:26.689491 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:14:26.689491 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:14:26.689491 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:14:26.689491 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:14:26.689491 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:14:26.693044 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 82/167] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jul 15 19:14:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:14:26.694301 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bee96f-7861-4bd0-8404-46ab74021e87 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:26.715702 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51d35c4-b515-40fa-962f-6284805ea6ab None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:26.720913 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:26.753639 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d95abad-09c5-4c35-8d49-fa6348171c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:26.754819 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95abad-09c5-4c35-8d49-fa6348171c13 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:26.755250 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95abad-09c5-4c35-8d49-fa6348171c13 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:26.755679 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95abad-09c5-4c35-8d49-fa6348171c13 None admin] PATH_INFO: `/v3/users/da619d0bd9ad4a48a65b481207b21979` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:26.757041 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d95abad-09c5-4c35-8d49-fa6348171c13 None admin] RBAC: Authorizing `identity:get_user(user_id=da619d0bd9ad4a48a65b481207b21979)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:26.759459 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d95abad-09c5-4c35-8d49-fa6348171c13 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:26.762244 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 86/168] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 15 19:14:26 2022] GET /identity/v3/users/da619d0bd9ad4a48a65b481207b21979 => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:26.763244 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d95abad-09c5-4c35-8d49-fa6348171c13 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:26.790257 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:26.809305 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bee96f-7861-4bd0-8404-46ab74021e87 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:26.844976 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f173b582-2b10-4013-b825-6316a4d40696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:26.846016 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f173b582-2b10-4013-b825-6316a4d40696 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:26.846425 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f173b582-2b10-4013-b825-6316a4d40696 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:26.846769 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f173b582-2b10-4013-b825-6316a4d40696 None admin] PATH_INFO: `/v3/users/da619d0bd9ad4a48a65b481207b21979` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:26.848249 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f173b582-2b10-4013-b825-6316a4d40696 None admin] RBAC: Authorizing `identity:get_user(user_id=da619d0bd9ad4a48a65b481207b21979)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:26.849882 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f173b582-2b10-4013-b825-6316a4d40696 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:26.851849 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 83/169] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 15 19:14:26 2022] GET /identity/v3/users/da619d0bd9ad4a48a65b481207b21979 => generated 275 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:26.852777 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f173b582-2b10-4013-b825-6316a4d40696 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:26.875414 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d95abad-09c5-4c35-8d49-fa6348171c13 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:26.913011 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f67c24-06ab-4b2d-8cd0-a1d33a66bbc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:26.914194 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f67c24-06ab-4b2d-8cd0-a1d33a66bbc4 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:26.914728 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f67c24-06ab-4b2d-8cd0-a1d33a66bbc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:26.915268 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f67c24-06ab-4b2d-8cd0-a1d33a66bbc4 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:26.916214 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f67c24-06ab-4b2d-8cd0-a1d33a66bbc4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:26.916718 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f67c24-06ab-4b2d-8cd0-a1d33a66bbc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:26.918471 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f67c24-06ab-4b2d-8cd0-a1d33a66bbc4 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:26.927149 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 87/170] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 15 19:14:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:26.928175 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f67c24-06ab-4b2d-8cd0-a1d33a66bbc4 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:26.960308 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:26.977734 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f173b582-2b10-4013-b825-6316a4d40696 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:27.010162 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:27.039066 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad4d4b5-0299-429b-b9ef-1d8630bc548b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:27.039759 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad4d4b5-0299-429b-b9ef-1d8630bc548b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:27.040038 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad4d4b5-0299-429b-b9ef-1d8630bc548b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:27.040363 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad4d4b5-0299-429b-b9ef-1d8630bc548b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:27.042961 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f67c24-06ab-4b2d-8cd0-a1d33a66bbc4 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:27.078833 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608e7cc0-3910-4c9d-b1a6-e6ad14f0a48d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:27.080105 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608e7cc0-3910-4c9d-b1a6-e6ad14f0a48d None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:27.080699 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608e7cc0-3910-4c9d-b1a6-e6ad14f0a48d None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:27.081285 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608e7cc0-3910-4c9d-b1a6-e6ad14f0a48d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:27.159242 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad4d4b5-0299-429b-b9ef-1d8630bc548b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:27.160758 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad4d4b5-0299-429b-b9ef-1d8630bc548b None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:27.161702 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608e7cc0-3910-4c9d-b1a6-e6ad14f0a48d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:27.163477 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608e7cc0-3910-4c9d-b1a6-e6ad14f0a48d None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:27.250205 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:27.298056 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 88/171] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 15 19:14:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 19:14:27.298970 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608e7cc0-3910-4c9d-b1a6-e6ad14f0a48d None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:27.318023 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:27.342716 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 84/172] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 15 19:14:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 19:14:27.343634 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad4d4b5-0299-429b-b9ef-1d8630bc548b None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:27.477695 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608e7cc0-3910-4c9d-b1a6-e6ad14f0a48d None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:27.538370 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54f8f99-1be3-4fa5-b02a-1cbc234d3f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:27.539378 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f8f99-1be3-4fa5-b02a-1cbc234d3f3d None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:27.539728 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f8f99-1be3-4fa5-b02a-1cbc234d3f3d None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:27.540034 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f8f99-1be3-4fa5-b02a-1cbc234d3f3d None admin] PATH_INFO: `/v3/users/96f47b71ea5a45ad92368f6b4130c36c` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:27.541343 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54f8f99-1be3-4fa5-b02a-1cbc234d3f3d None admin] RBAC: Authorizing `identity:get_user(user_id=96f47b71ea5a45ad92368f6b4130c36c)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:27.546380 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54f8f99-1be3-4fa5-b02a-1cbc234d3f3d None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:27.548288 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 89/173] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 15 19:14:27 2022] GET /identity/v3/users/96f47b71ea5a45ad92368f6b4130c36c => generated 247 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:27.548835 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54f8f99-1be3-4fa5-b02a-1cbc234d3f3d None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:27.589074 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad4d4b5-0299-429b-b9ef-1d8630bc548b None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:27.598136 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:27.655493 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f56b2c-0205-40d8-ad01-c90b57ae5b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:27.656385 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f56b2c-0205-40d8-ad01-c90b57ae5b23 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:27.656666 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f56b2c-0205-40d8-ad01-c90b57ae5b23 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:27.656939 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f56b2c-0205-40d8-ad01-c90b57ae5b23 None admin] PATH_INFO: `/v3/projects/e733f93067e74ac193989bcd00d8d5ce` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:27.661358 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54f8f99-1be3-4fa5-b02a-1cbc234d3f3d None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:27.662564 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f56b2c-0205-40d8-ad01-c90b57ae5b23 None admin] RBAC: Authorizing `identity:get_project(project_id=e733f93067e74ac193989bcd00d8d5ce)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:27.664066 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f56b2c-0205-40d8-ad01-c90b57ae5b23 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:27.666523 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 85/174] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 15 19:14:27 2022] GET /identity/v3/projects/e733f93067e74ac193989bcd00d8d5ce => generated 312 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:27.667159 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f56b2c-0205-40d8-ad01-c90b57ae5b23 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:27.698807 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b1f060-a05f-4eea-8bba-35743cd0091b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:27.699778 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b1f060-a05f-4eea-8bba-35743cd0091b None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:27.700065 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b1f060-a05f-4eea-8bba-35743cd0091b None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:27.700343 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b1f060-a05f-4eea-8bba-35743cd0091b None admin] PATH_INFO: `/v3/projects/513641c037c84c54b779b5ca0684f17c` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:27.701563 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b1f060-a05f-4eea-8bba-35743cd0091b None admin] RBAC: Authorizing `identity:get_project(project_id=513641c037c84c54b779b5ca0684f17c)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:27.703133 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b1f060-a05f-4eea-8bba-35743cd0091b None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:27.705012 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 90/175] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 15 19:14:27 2022] GET /identity/v3/projects/513641c037c84c54b779b5ca0684f17c => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:27.705717 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b1f060-a05f-4eea-8bba-35743cd0091b None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:27.763392 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:27.813199 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b1f060-a05f-4eea-8bba-35743cd0091b None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:27.831342 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f56b2c-0205-40d8-ad01-c90b57ae5b23 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:27.850130 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:27.854270 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19887fb1-a71c-405f-98d3-062671f3cced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:27.855093 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19887fb1-a71c-405f-98d3-062671f3cced None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:27.855380 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19887fb1-a71c-405f-98d3-062671f3cced None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:27.855658 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19887fb1-a71c-405f-98d3-062671f3cced None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:27.857164 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19887fb1-a71c-405f-98d3-062671f3cced None admin] RBAC: Adding query filter params (role.id=28763c6e70094116b3a4b2895895fa58, scope.project.id=e733f93067e74ac193989bcd00d8d5ce, user.id=da619d0bd9ad4a48a65b481207b21979) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:27.857480 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19887fb1-a71c-405f-98d3-062671f3cced None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:27.859521 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19887fb1-a71c-405f-98d3-062671f3cced None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:27.867693 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 91/176] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 15 19:14:27 2022] GET /identity/v3/role_assignments?user.id=da619d0bd9ad4a48a65b481207b21979&scope.project.id=e733f93067e74ac193989bcd00d8d5ce&role.id=28763c6e70094116b3a4b2895895fa58 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:27.868823 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a87e645-a2f3-465d-ace1-48031f9ed3ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:27.868979 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19887fb1-a71c-405f-98d3-062671f3cced None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:27.869705 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a87e645-a2f3-465d-ace1-48031f9ed3ad None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:27.870072 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a87e645-a2f3-465d-ace1-48031f9ed3ad None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:27.870415 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a87e645-a2f3-465d-ace1-48031f9ed3ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:27.953489 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a87e645-a2f3-465d-ace1-48031f9ed3ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:27.955170 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a87e645-a2f3-465d-ace1-48031f9ed3ad None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:28.011581 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19887fb1-a71c-405f-98d3-062671f3cced None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:28.035543 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:28.057675 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d957f679-22e4-40a4-9f5f-ab5fb05336ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:28.058293 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d957f679-22e4-40a4-9f5f-ab5fb05336ef None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:28.058607 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d957f679-22e4-40a4-9f5f-ab5fb05336ef None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:28.059149 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d957f679-22e4-40a4-9f5f-ab5fb05336ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:28.059721 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d957f679-22e4-40a4-9f5f-ab5fb05336ef None admin] RBAC: Adding query filter params (role.id=e97d26621445466d9550261c608db9c5, scope.project.id=513641c037c84c54b779b5ca0684f17c, user.id=da619d0bd9ad4a48a65b481207b21979) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:28.060042 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d957f679-22e4-40a4-9f5f-ab5fb05336ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:28.061560 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d957f679-22e4-40a4-9f5f-ab5fb05336ef None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:28.068528 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 92/177] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 15 19:14:27 2022] GET /identity/v3/role_assignments?user.id=da619d0bd9ad4a48a65b481207b21979&scope.project.id=513641c037c84c54b779b5ca0684f17c&role.id=e97d26621445466d9550261c608db9c5 => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:28.069348 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ab01e-3a59-4c76-8a9d-c835c7b1334f None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:28.069646 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ab01e-3a59-4c76-8a9d-c835c7b1334f None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:28.069953 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ab01e-3a59-4c76-8a9d-c835c7b1334f None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:28.071880 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 93/178] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:14:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:14:28.075430 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ded711-6ca1-4a0b-9036-b0a5e4624f4d None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:28.076275 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ded711-6ca1-4a0b-9036-b0a5e4624f4d None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:28.077006 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ded711-6ca1-4a0b-9036-b0a5e4624f4d None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:28.078705 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 94/179] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:14:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:14:28.084106 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440f28f-18ad-404e-b008-71fea2bc5821 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:28.084412 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440f28f-18ad-404e-b008-71fea2bc5821 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:28.084847 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440f28f-18ad-404e-b008-71fea2bc5821 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:28.086341 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 95/180] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:14:28 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:14:28.088059 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 86/181] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 15 19:14:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 19:14:28.088735 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a19ea3-34f4-4fbd-9fd4-1c927a4bbce3 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:28.089256 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a19ea3-34f4-4fbd-9fd4-1c927a4bbce3 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:28.089505 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c3e201-23e1-4fd0-bae0-8f1ed1a2888d None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:28.089826 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c3e201-23e1-4fd0-bae0-8f1ed1a2888d None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:28.090109 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c3e201-23e1-4fd0-bae0-8f1ed1a2888d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:28.090404 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a19ea3-34f4-4fbd-9fd4-1c927a4bbce3 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:28.092241 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 96/182] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:14:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:14:28.093890 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a1366e-5b4d-4df1-ae52-29349ab401ba None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:28.094384 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a1366e-5b4d-4df1-ae52-29349ab401ba None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:28.094862 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a1366e-5b4d-4df1-ae52-29349ab401ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:28.109649 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-37c3e201-23e1-4fd0-bae0-8f1ed1a2888d None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:14:28.110961 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-37c3e201-23e1-4fd0-bae0-8f1ed1a2888d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:14:28.112872 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-31a1366e-5b4d-4df1-ae52-29349ab401ba None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:14:28.114123 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-31a1366e-5b4d-4df1-ae52-29349ab401ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:14:28.158864 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:28.213790 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 87/183] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:14:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:14:28.213790 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1bade2-9d92-4cd6-aa3d-59ef72bc696f None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:28.213790 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1bade2-9d92-4cd6-aa3d-59ef72bc696f None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:28.213790 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1bade2-9d92-4cd6-aa3d-59ef72bc696f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:28.220390 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 97/184] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:14:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:14:28.221605 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a1366e-5b4d-4df1-ae52-29349ab401ba None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:28.235958 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-4d1bade2-9d92-4cd6-aa3d-59ef72bc696f None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:14:28.237197 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-4d1bade2-9d92-4cd6-aa3d-59ef72bc696f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:14:28.330145 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 88/185] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:14:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:14:28.331523 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebccf841-4f30-4cbe-b932-9ce820d8c7df None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:28.331825 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebccf841-4f30-4cbe-b932-9ce820d8c7df None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:28.332098 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebccf841-4f30-4cbe-b932-9ce820d8c7df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:28.348395 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-ebccf841-4f30-4cbe-b932-9ce820d8c7df None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:14:28.354699 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-ebccf841-4f30-4cbe-b932-9ce820d8c7df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:14:28.354823 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a1366e-5b4d-4df1-ae52-29349ab401ba None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:28.361641 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:28.386903 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313fa9e6-2d78-4c3a-a757-11b31df30bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:28.387770 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313fa9e6-2d78-4c3a-a757-11b31df30bd4 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:28.388055 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313fa9e6-2d78-4c3a-a757-11b31df30bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:28.388329 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313fa9e6-2d78-4c3a-a757-11b31df30bd4 None admin] PATH_INFO: `/v3/projects/513641c037c84c54b779b5ca0684f17c` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:28.389519 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313fa9e6-2d78-4c3a-a757-11b31df30bd4 None admin] RBAC: Authorizing `identity:get_project(project_id=513641c037c84c54b779b5ca0684f17c)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:28.391023 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313fa9e6-2d78-4c3a-a757-11b31df30bd4 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:28.392785 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 98/186] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 15 19:14:28 2022] GET /identity/v3/projects/513641c037c84c54b779b5ca0684f17c => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:28.393547 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313fa9e6-2d78-4c3a-a757-11b31df30bd4 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:28.425084 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 89/187] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:14:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:14:28.426322 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebccf841-4f30-4cbe-b932-9ce820d8c7df None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:28.430638 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:28.499340 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313fa9e6-2d78-4c3a-a757-11b31df30bd4 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:28.525909 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebccf841-4f30-4cbe-b932-9ce820d8c7df None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:28.536498 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fab6efb-16a1-4197-afdd-c616a03b84f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:28.537569 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fab6efb-16a1-4197-afdd-c616a03b84f1 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:28.537885 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fab6efb-16a1-4197-afdd-c616a03b84f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:28.538162 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fab6efb-16a1-4197-afdd-c616a03b84f1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:28.548304 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fab6efb-16a1-4197-afdd-c616a03b84f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:14:28.556557 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-1fab6efb-16a1-4197-afdd-c616a03b84f1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:14:28.556557 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:14:28.556557 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:14:28.556557 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:14:28.556557 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:14:28.556557 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:14:28.556557 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:14:28.556557 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:14:28.556557 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:14:28.556557 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:14:28.556557 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:14:28.556557 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:14:28.556557 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:14:28.556557 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:14:28.556557 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:14:28.556557 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:14:28.557424 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:14:28.557424 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:14:28.557424 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:14:28.557424 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:14:28.557424 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:14:28.557424 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:14:28.557424 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:14:28.557424 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:14:28.557424 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:14:28.557424 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:14:28.557424 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:14:28.557424 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:14:28.557424 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:14:28.557424 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:14:28.557424 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:14:28.557424 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:14:28.558265 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:14:28.558265 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:14:28.558265 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:14:28.558265 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:14:28.558265 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:14:28.558265 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:14:28.558265 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:14:28.558265 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:14:28.558265 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:14:28.558265 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:14:28.558265 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:14:28.558265 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:14:28.558265 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 99/188] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jul 15 19:14:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:14:28.558911 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fab6efb-16a1-4197-afdd-c616a03b84f1 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:28.562772 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabf5ef0-fabd-4185-8f46-0b47d1807cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:28.563255 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf5ef0-fabd-4185-8f46-0b47d1807cde None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:28.563536 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf5ef0-fabd-4185-8f46-0b47d1807cde None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:28.563809 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf5ef0-fabd-4185-8f46-0b47d1807cde None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:28.574950 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabf5ef0-fabd-4185-8f46-0b47d1807cde None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:14:28.582582 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-aabf5ef0-fabd-4185-8f46-0b47d1807cde None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 19:14:28.582582 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:14:28.582582 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:14:28.582582 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:14:28.582582 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:14:28.582582 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:14:28.582582 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:14:28.582582 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:14:28.582582 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:14:28.582582 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:14:28.582582 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:14:28.582582 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:14:28.582582 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:14:28.582582 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:14:28.582582 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:14:28.582582 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:14:28.583831 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:14:28.583831 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:14:28.583831 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:14:28.583831 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:14:28.583831 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:14:28.583831 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:14:28.583831 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:14:28.583831 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:14:28.583831 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:14:28.583831 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:14:28.583831 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:14:28.583831 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:14:28.583831 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:14:28.583831 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:14:28.583831 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:14:28.583831 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:14:28.584922 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:14:28.584922 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:14:28.584922 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:14:28.584922 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:14:28.584922 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:14:28.584922 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:14:28.584922 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:14:28.584922 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:14:28.584922 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:14:28.584922 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:14:28.584922 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 19:14:28.584922 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:14:28.584922 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 90/189] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 15 19:14:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:14:28.584922 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabf5ef0-fabd-4185-8f46-0b47d1807cde None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:28.593466 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:28.665869 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:28.672541 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fab6efb-16a1-4197-afdd-c616a03b84f1 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:28.685189 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabf5ef0-fabd-4185-8f46-0b47d1807cde None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:28.705905 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37453e85-b7d8-4076-ae9c-78f63178ab72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:28.706700 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37453e85-b7d8-4076-ae9c-78f63178ab72 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:28.706984 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37453e85-b7d8-4076-ae9c-78f63178ab72 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:28.707261 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37453e85-b7d8-4076-ae9c-78f63178ab72 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:28.708446 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37453e85-b7d8-4076-ae9c-78f63178ab72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:28.709910 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37453e85-b7d8-4076-ae9c-78f63178ab72 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:28.711551 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 100/190] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:14:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:28.712310 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37453e85-b7d8-4076-ae9c-78f63178ab72 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:28.722830 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ad5af8-d254-494c-ac72-5c73759d0983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:28.723495 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ad5af8-d254-494c-ac72-5c73759d0983 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:28.724092 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ad5af8-d254-494c-ac72-5c73759d0983 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:28.724374 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ad5af8-d254-494c-ac72-5c73759d0983 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:28.725219 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ad5af8-d254-494c-ac72-5c73759d0983 None admin] RBAC: Adding query filter params (role.id=4318f86ba9f74cd4a0bef1c311276657, scope.project.id=513641c037c84c54b779b5ca0684f17c, user.id=96f47b71ea5a45ad92368f6b4130c36c) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:28.725532 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ad5af8-d254-494c-ac72-5c73759d0983 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:28.727563 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ad5af8-d254-494c-ac72-5c73759d0983 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:28.733444 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 91/191] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 15 19:14:28 2022] GET /identity/v3/role_assignments?user.id=96f47b71ea5a45ad92368f6b4130c36c&scope.project.id=513641c037c84c54b779b5ca0684f17c&role.id=4318f86ba9f74cd4a0bef1c311276657 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:28.734271 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ad5af8-d254-494c-ac72-5c73759d0983 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:28.827825 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37453e85-b7d8-4076-ae9c-78f63178ab72 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:28.828860 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:28.864273 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21e563f-d36a-4ce7-93ff-341e427a3b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:28.865000 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21e563f-d36a-4ce7-93ff-341e427a3b46 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:28.865279 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21e563f-d36a-4ce7-93ff-341e427a3b46 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:28.865551 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21e563f-d36a-4ce7-93ff-341e427a3b46 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:28.870657 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ad5af8-d254-494c-ac72-5c73759d0983 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:28.877577 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21e563f-d36a-4ce7-93ff-341e427a3b46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:14:28.885881 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-d21e563f-d36a-4ce7-93ff-341e427a3b46 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 19:14:28.885881 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:14:28.885881 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:14:28.885881 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:14:28.885881 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:14:28.885881 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:14:28.885881 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:14:28.885881 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:14:28.885881 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:14:28.885881 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:14:28.885881 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:14:28.885881 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:14:28.885881 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:14:28.885881 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:14:28.885881 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:14:28.885881 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:14:28.886846 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:14:28.886846 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:14:28.886846 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:14:28.886846 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:14:28.886846 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:14:28.886846 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:14:28.886846 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:14:28.886846 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:14:28.886846 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:14:28.886846 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:14:28.886846 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:14:28.886846 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:14:28.886846 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:14:28.886846 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:14:28.886846 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:14:28.886846 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:14:28.887643 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:14:28.887643 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:14:28.887643 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:14:28.887643 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:14:28.887643 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:14:28.887643 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:14:28.887643 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:14:28.887643 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:14:28.887643 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:14:28.887643 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:14:28.887643 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 19:14:28.887643 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:14:28.887643 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 101/192] 10.222.0.33 () {56 vars in 1114 bytes} [Fri Jul 15 19:14:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:14:28.888350 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21e563f-d36a-4ce7-93ff-341e427a3b46 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:28.915577 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:28.919041 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4fd390-a709-4401-b003-1b976f478779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:28.920099 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4fd390-a709-4401-b003-1b976f478779 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:28.920099 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4fd390-a709-4401-b003-1b976f478779 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:28.920287 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4fd390-a709-4401-b003-1b976f478779 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:28.921136 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4fd390-a709-4401-b003-1b976f478779 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:28.921444 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4fd390-a709-4401-b003-1b976f478779 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:28.922859 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4fd390-a709-4401-b003-1b976f478779 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:28.933635 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 92/193] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 15 19:14:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:28.934276 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4fd390-a709-4401-b003-1b976f478779 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:28.995401 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21e563f-d36a-4ce7-93ff-341e427a3b46 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:29.031070 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c44b22e-2928-47f8-95bd-561481823ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:29.031737 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c44b22e-2928-47f8-95bd-561481823ebc None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:29.032017 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c44b22e-2928-47f8-95bd-561481823ebc None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:29.032290 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c44b22e-2928-47f8-95bd-561481823ebc None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:29.033035 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c44b22e-2928-47f8-95bd-561481823ebc None admin] RBAC: Adding query filter params (name=reader) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:29.033337 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c44b22e-2928-47f8-95bd-561481823ebc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:29.034750 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c44b22e-2928-47f8-95bd-561481823ebc None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:29.041943 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4fd390-a709-4401-b003-1b976f478779 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:29.043410 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 102/194] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jul 15 19:14:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:29.048257 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c44b22e-2928-47f8-95bd-561481823ebc None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:29.075729 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:29.080562 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bba534d-45aa-4de0-b826-cd02fb413b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:29.081237 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bba534d-45aa-4de0-b826-cd02fb413b85 None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:29.091354 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bba534d-45aa-4de0-b826-cd02fb413b85 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:29.091354 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bba534d-45aa-4de0-b826-cd02fb413b85 None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:29.091354 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bba534d-45aa-4de0-b826-cd02fb413b85 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:29.091354 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bba534d-45aa-4de0-b826-cd02fb413b85 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:29.131542 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:29.146558 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c44b22e-2928-47f8-95bd-561481823ebc None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:29.179062 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387b32f9-9a03-470d-a8eb-0acd831c2b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:29.179737 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387b32f9-9a03-470d-a8eb-0acd831c2b22 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:29.180012 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387b32f9-9a03-470d-a8eb-0acd831c2b22 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:29.180286 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387b32f9-9a03-470d-a8eb-0acd831c2b22 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:29.181024 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387b32f9-9a03-470d-a8eb-0acd831c2b22 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:29.181328 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387b32f9-9a03-470d-a8eb-0acd831c2b22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:29.182724 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387b32f9-9a03-470d-a8eb-0acd831c2b22 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:29.190294 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 103/195] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 15 19:14:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:29.191069 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387b32f9-9a03-470d-a8eb-0acd831c2b22 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:29.204712 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 93/196] 10.222.0.33 () {60 vars in 1132 bytes} [Fri Jul 15 19:14:28 2022] POST /identity/v3/users => generated 297 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:14:29.217992 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bba534d-45aa-4de0-b826-cd02fb413b85 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:29.304974 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387b32f9-9a03-470d-a8eb-0acd831c2b22 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:29.324925 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bba534d-45aa-4de0-b826-cd02fb413b85 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:29.344544 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3635db34-fd75-4925-b759-4e016bf7111a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:29.345435 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3635db34-fd75-4925-b759-4e016bf7111a None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:29.345714 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3635db34-fd75-4925-b759-4e016bf7111a None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:29.346012 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3635db34-fd75-4925-b759-4e016bf7111a None admin] PATH_INFO: `/v3/users/085b688691184e23a0ee3794af80b0e5` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:29.363659 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3635db34-fd75-4925-b759-4e016bf7111a None admin] RBAC: Authorizing `identity:get_user(user_id=085b688691184e23a0ee3794af80b0e5)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:29.365116 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3635db34-fd75-4925-b759-4e016bf7111a None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:29.367057 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 104/197] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 15 19:14:29 2022] GET /identity/v3/users/085b688691184e23a0ee3794af80b0e5 => generated 283 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:29.367293 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08a2487-33ff-4562-8c21-52e27a44543a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:29.367765 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3635db34-fd75-4925-b759-4e016bf7111a None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:29.368249 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08a2487-33ff-4562-8c21-52e27a44543a None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:29.368535 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08a2487-33ff-4562-8c21-52e27a44543a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:29.368812 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08a2487-33ff-4562-8c21-52e27a44543a None admin] PATH_INFO: `/v3/users/0505801f23f1420eb449bad3d0b36bec` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:29.370129 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08a2487-33ff-4562-8c21-52e27a44543a None admin] RBAC: Authorizing `identity:get_user(user_id=0505801f23f1420eb449bad3d0b36bec)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:29.371824 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08a2487-33ff-4562-8c21-52e27a44543a None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:29.373513 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:29.373847 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 94/198] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 15 19:14:29 2022] GET /identity/v3/users/0505801f23f1420eb449bad3d0b36bec => generated 289 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:29.374601 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08a2487-33ff-4562-8c21-52e27a44543a None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:29.452979 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:29.499552 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3635db34-fd75-4925-b759-4e016bf7111a None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:29.505925 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08a2487-33ff-4562-8c21-52e27a44543a None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:29.541756 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2bb3df-77d1-42ad-8872-7e7c6ef79fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:29.543274 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2bb3df-77d1-42ad-8872-7e7c6ef79fed None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:29.543562 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2bb3df-77d1-42ad-8872-7e7c6ef79fed None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:29.543838 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2bb3df-77d1-42ad-8872-7e7c6ef79fed None admin] PATH_INFO: `/v3/users/085b688691184e23a0ee3794af80b0e5` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:29.545055 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2bb3df-77d1-42ad-8872-7e7c6ef79fed None admin] RBAC: Authorizing `identity:get_user(user_id=085b688691184e23a0ee3794af80b0e5)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:29.546476 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2bb3df-77d1-42ad-8872-7e7c6ef79fed None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:29.546938 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f43a1d7-a68f-46a4-a21d-e1e3e9fa753a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:29.547594 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f43a1d7-a68f-46a4-a21d-e1e3e9fa753a None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:29.547875 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f43a1d7-a68f-46a4-a21d-e1e3e9fa753a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:29.548274 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 105/199] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 15 19:14:29 2022] GET /identity/v3/users/085b688691184e23a0ee3794af80b0e5 => generated 283 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:29.548376 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f43a1d7-a68f-46a4-a21d-e1e3e9fa753a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:29.548941 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2bb3df-77d1-42ad-8872-7e7c6ef79fed None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:29.640606 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:29.642017 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f43a1d7-a68f-46a4-a21d-e1e3e9fa753a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:29.643719 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f43a1d7-a68f-46a4-a21d-e1e3e9fa753a None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:29.676694 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2bb3df-77d1-42ad-8872-7e7c6ef79fed None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:29.723112 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:29.731491 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5490c6b-6570-4226-b78c-2e66bdce4140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:29.732651 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5490c6b-6570-4226-b78c-2e66bdce4140 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:29.733101 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5490c6b-6570-4226-b78c-2e66bdce4140 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:29.733591 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5490c6b-6570-4226-b78c-2e66bdce4140 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:29.771775 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 95/200] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 15 19:14:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 19:14:29.772721 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f43a1d7-a68f-46a4-a21d-e1e3e9fa753a None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:29.834386 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5490c6b-6570-4226-b78c-2e66bdce4140 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:29.836568 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5490c6b-6570-4226-b78c-2e66bdce4140 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:29.882956 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f43a1d7-a68f-46a4-a21d-e1e3e9fa753a None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:29.927859 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:29.932893 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920c395e-23f9-4703-bdcb-34d695bf2431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:29.933684 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c395e-23f9-4703-bdcb-34d695bf2431 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:29.934063 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c395e-23f9-4703-bdcb-34d695bf2431 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:29.934411 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c395e-23f9-4703-bdcb-34d695bf2431 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:29.961262 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 106/201] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 15 19:14:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 19:14:29.962627 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5490c6b-6570-4226-b78c-2e66bdce4140 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:29.972499 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:30.027599 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920c395e-23f9-4703-bdcb-34d695bf2431 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:30.029345 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920c395e-23f9-4703-bdcb-34d695bf2431 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:30.080166 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5490c6b-6570-4226-b78c-2e66bdce4140 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:30.118974 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232e67cd-000a-4a8d-96a2-15684e3504ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:30.120256 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232e67cd-000a-4a8d-96a2-15684e3504ba None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:30.120696 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232e67cd-000a-4a8d-96a2-15684e3504ba None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:30.121170 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232e67cd-000a-4a8d-96a2-15684e3504ba None admin] PATH_INFO: `/v3/projects/12e6f4bc187942989f83aeb1902a3036` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:30.122775 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232e67cd-000a-4a8d-96a2-15684e3504ba None admin] RBAC: Authorizing `identity:get_project(project_id=12e6f4bc187942989f83aeb1902a3036)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:30.124467 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232e67cd-000a-4a8d-96a2-15684e3504ba None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:30.126475 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 107/202] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 15 19:14:29 2022] GET /identity/v3/projects/12e6f4bc187942989f83aeb1902a3036 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:30.132631 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232e67cd-000a-4a8d-96a2-15684e3504ba None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:30.164224 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 96/203] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 15 19:14:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 19:14:30.165619 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920c395e-23f9-4703-bdcb-34d695bf2431 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:30.182375 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:30.223099 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:30.249799 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232e67cd-000a-4a8d-96a2-15684e3504ba None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:30.272821 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920c395e-23f9-4703-bdcb-34d695bf2431 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:30.290446 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2457507-f564-46ee-8056-7ab95342c8b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:30.291777 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2457507-f564-46ee-8056-7ab95342c8b9 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:30.292282 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2457507-f564-46ee-8056-7ab95342c8b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:30.292753 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2457507-f564-46ee-8056-7ab95342c8b9 None admin] PATH_INFO: `/v3/projects/513641c037c84c54b779b5ca0684f17c` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:30.294247 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2457507-f564-46ee-8056-7ab95342c8b9 None admin] RBAC: Authorizing `identity:get_project(project_id=513641c037c84c54b779b5ca0684f17c)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:30.296087 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2457507-f564-46ee-8056-7ab95342c8b9 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:30.302093 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 108/204] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 15 19:14:30 2022] GET /identity/v3/projects/513641c037c84c54b779b5ca0684f17c => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:30.303286 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2457507-f564-46ee-8056-7ab95342c8b9 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:30.309795 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddab4c5e-74e1-46cc-b47e-057e561a4ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:30.310346 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddab4c5e-74e1-46cc-b47e-057e561a4ca7 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:30.310457 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddab4c5e-74e1-46cc-b47e-057e561a4ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:30.310694 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddab4c5e-74e1-46cc-b47e-057e561a4ca7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:30.316525 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddab4c5e-74e1-46cc-b47e-057e561a4ca7 None admin] RBAC: Adding query filter params (role.id=4318f86ba9f74cd4a0bef1c311276657, scope.project.id=12e6f4bc187942989f83aeb1902a3036, user.id=085b688691184e23a0ee3794af80b0e5) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:30.316525 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddab4c5e-74e1-46cc-b47e-057e561a4ca7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:30.316525 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddab4c5e-74e1-46cc-b47e-057e561a4ca7 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:30.323936 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 97/205] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 15 19:14:30 2022] GET /identity/v3/role_assignments?user.id=085b688691184e23a0ee3794af80b0e5&scope.project.id=12e6f4bc187942989f83aeb1902a3036&role.id=4318f86ba9f74cd4a0bef1c311276657 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:30.323936 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddab4c5e-74e1-46cc-b47e-057e561a4ca7 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:30.439612 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2457507-f564-46ee-8056-7ab95342c8b9 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:30.451168 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:30.481101 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1fbcfe-9ecb-4a4d-9419-976ee7af070c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:30.482404 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1fbcfe-9ecb-4a4d-9419-976ee7af070c None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:30.482899 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1fbcfe-9ecb-4a4d-9419-976ee7af070c None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:30.483403 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1fbcfe-9ecb-4a4d-9419-976ee7af070c None admin] PATH_INFO: `/v3/projects/12e6f4bc187942989f83aeb1902a3036` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:30.485000 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1fbcfe-9ecb-4a4d-9419-976ee7af070c None admin] RBAC: Authorizing `identity:get_project(project_id=12e6f4bc187942989f83aeb1902a3036)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:30.486827 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1fbcfe-9ecb-4a4d-9419-976ee7af070c None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:30.489406 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 109/206] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 15 19:14:30 2022] GET /identity/v3/projects/12e6f4bc187942989f83aeb1902a3036 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:30.495840 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1fbcfe-9ecb-4a4d-9419-976ee7af070c None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:30.542558 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddab4c5e-74e1-46cc-b47e-057e561a4ca7 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:30.585300 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:30.629317 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fe41f3-29c5-42fa-beff-50acb6b9ef35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:30.630425 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fe41f3-29c5-42fa-beff-50acb6b9ef35 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:30.630919 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fe41f3-29c5-42fa-beff-50acb6b9ef35 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:30.632220 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fe41f3-29c5-42fa-beff-50acb6b9ef35 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:30.633409 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fe41f3-29c5-42fa-beff-50acb6b9ef35 None admin] RBAC: Adding query filter params (role.id=2f62da1ebb2b443eabf21e910a215048, scope.project.id=513641c037c84c54b779b5ca0684f17c, user.id=0505801f23f1420eb449bad3d0b36bec) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:30.634158 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fe41f3-29c5-42fa-beff-50acb6b9ef35 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:30.636486 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fe41f3-29c5-42fa-beff-50acb6b9ef35 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:30.648222 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 98/207] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 15 19:14:30 2022] GET /identity/v3/role_assignments?user.id=0505801f23f1420eb449bad3d0b36bec&scope.project.id=513641c037c84c54b779b5ca0684f17c&role.id=2f62da1ebb2b443eabf21e910a215048 => generated 608 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:30.656019 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1fbcfe-9ecb-4a4d-9419-976ee7af070c None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:30.710561 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4413676c-1ae0-46fe-806d-0b4416cd4d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:30.712184 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4413676c-1ae0-46fe-806d-0b4416cd4d5b None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:30.712686 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4413676c-1ae0-46fe-806d-0b4416cd4d5b None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:30.713165 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4413676c-1ae0-46fe-806d-0b4416cd4d5b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:30.714289 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4413676c-1ae0-46fe-806d-0b4416cd4d5b None admin] RBAC: Adding query filter params (role.id=e97d26621445466d9550261c608db9c5, scope.project.id=12e6f4bc187942989f83aeb1902a3036, user.id=085b688691184e23a0ee3794af80b0e5) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:30.714808 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4413676c-1ae0-46fe-806d-0b4416cd4d5b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:30.716457 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4413676c-1ae0-46fe-806d-0b4416cd4d5b None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:30.726424 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 110/208] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 15 19:14:30 2022] GET /identity/v3/role_assignments?user.id=085b688691184e23a0ee3794af80b0e5&scope.project.id=12e6f4bc187942989f83aeb1902a3036&role.id=e97d26621445466d9550261c608db9c5 => generated 259 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:14:31.436033 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fbe829-f0f6-4915-a8a4-021314922800 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:31.436547 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fbe829-f0f6-4915-a8a4-021314922800 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:31.436621 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fbe829-f0f6-4915-a8a4-021314922800 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:31.438263 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 99/209] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:14:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:14:31.443815 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9e60ef-a948-4fe8-9129-9e9ac06dbdac None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:31.444216 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9e60ef-a948-4fe8-9129-9e9ac06dbdac None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:31.444408 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9e60ef-a948-4fe8-9129-9e9ac06dbdac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:31.461590 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-dd9e60ef-a948-4fe8-9129-9e9ac06dbdac None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:14:31.462831 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-dd9e60ef-a948-4fe8-9129-9e9ac06dbdac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:14:31.498595 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:31.532402 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 111/210] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:14:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:14:31.555640 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fbe829-f0f6-4915-a8a4-021314922800 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:31.660754 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34026ac8-d9c1-4dda-a187-76b89cfd11d4 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:31.661074 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34026ac8-d9c1-4dda-a187-76b89cfd11d4 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:31.661316 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34026ac8-d9c1-4dda-a187-76b89cfd11d4 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:31.663027 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 112/211] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:14:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 19:14:31.668709 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11005e54-2b05-4664-9710-c5fbd6a9f3ce None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:31.669268 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11005e54-2b05-4664-9710-c5fbd6a9f3ce None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:31.669442 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fbe829-f0f6-4915-a8a4-021314922800 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:31.669942 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11005e54-2b05-4664-9710-c5fbd6a9f3ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:31.692439 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-11005e54-2b05-4664-9710-c5fbd6a9f3ce None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:14:31.693550 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-11005e54-2b05-4664-9710-c5fbd6a9f3ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:14:31.712648 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7b56d2-49b7-4247-a211-6f5047a57ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:31.713455 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7b56d2-49b7-4247-a211-6f5047a57ffc None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:31.713733 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7b56d2-49b7-4247-a211-6f5047a57ffc None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:31.714031 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7b56d2-49b7-4247-a211-6f5047a57ffc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:31.728874 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7b56d2-49b7-4247-a211-6f5047a57ffc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:14:31.738232 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-8b7b56d2-49b7-4247-a211-6f5047a57ffc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 19:14:31.738232 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:14:31.738232 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:14:31.738232 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:14:31.738232 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:14:31.738232 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:14:31.738232 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:14:31.738232 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:14:31.738232 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:14:31.738232 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:14:31.738232 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:14:31.738232 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:14:31.738232 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:14:31.738232 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:14:31.738232 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:14:31.738232 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:14:31.739120 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:14:31.739120 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:14:31.739120 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:14:31.739120 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:14:31.739120 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:14:31.739120 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:14:31.739120 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:14:31.739120 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:14:31.739120 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:14:31.739120 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:14:31.739120 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:14:31.739120 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:14:31.739120 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:14:31.739120 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:14:31.739120 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:14:31.739120 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:14:31.739946 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:14:31.739946 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:14:31.739946 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:14:31.739946 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:14:31.739946 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:14:31.739946 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:14:31.739946 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:14:31.739946 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:14:31.739946 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:14:31.739946 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:14:31.739946 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 19:14:31.739946 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:14:31.739946 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 100/212] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 15 19:14:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:14:31.746020 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7b56d2-49b7-4247-a211-6f5047a57ffc None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:31.785980 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 113/213] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:14:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:14:31.812714 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:31.813673 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11005e54-2b05-4664-9710-c5fbd6a9f3ce None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:31.883023 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7b56d2-49b7-4247-a211-6f5047a57ffc None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:31.934291 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7bbdc0-5113-47ce-8650-4809c56a56ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:31.935432 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7bbdc0-5113-47ce-8650-4809c56a56ee None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:31.936165 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7bbdc0-5113-47ce-8650-4809c56a56ee None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:31.936629 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7bbdc0-5113-47ce-8650-4809c56a56ee None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:31.938569 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7bbdc0-5113-47ce-8650-4809c56a56ee None admin] RBAC: Adding query filter params (name=member) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:31.939119 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7bbdc0-5113-47ce-8650-4809c56a56ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:31.941119 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:31.942723 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7bbdc0-5113-47ce-8650-4809c56a56ee None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:31.944391 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11005e54-2b05-4664-9710-c5fbd6a9f3ce None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:31.953358 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 101/214] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jul 15 19:14:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:31.959203 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7bbdc0-5113-47ce-8650-4809c56a56ee None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:31.988721 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffbfc6e-7e64-4070-9b72-0638ccd642e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:31.989990 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffbfc6e-7e64-4070-9b72-0638ccd642e7 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:31.990449 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffbfc6e-7e64-4070-9b72-0638ccd642e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:31.990893 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffbfc6e-7e64-4070-9b72-0638ccd642e7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:31.992350 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffbfc6e-7e64-4070-9b72-0638ccd642e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:31.995158 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffbfc6e-7e64-4070-9b72-0638ccd642e7 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:32.001851 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 114/215] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:14:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:14:32.003370 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffbfc6e-7e64-4070-9b72-0638ccd642e7 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:32.066290 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7bbdc0-5113-47ce-8650-4809c56a56ee None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:32.073507 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:32.101926 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b063eced-08d5-451d-9268-9df3cf353116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:32.105578 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b063eced-08d5-451d-9268-9df3cf353116 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:32.105578 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b063eced-08d5-451d-9268-9df3cf353116 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:32.105578 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b063eced-08d5-451d-9268-9df3cf353116 None admin] PATH_INFO: `/v3/users/605a9210f12a47ec8f5769bd86b7c636` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:32.113924 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffbfc6e-7e64-4070-9b72-0638ccd642e7 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:32.118092 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b063eced-08d5-451d-9268-9df3cf353116 None admin] RBAC: Authorizing `identity:get_user(user_id=605a9210f12a47ec8f5769bd86b7c636)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:32.119550 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b063eced-08d5-451d-9268-9df3cf353116 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:32.121373 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 102/216] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 15 19:14:31 2022] GET /identity/v3/users/605a9210f12a47ec8f5769bd86b7c636 => generated 297 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:14:32.126923 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b063eced-08d5-451d-9268-9df3cf353116 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:32.153554 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0f738f-5b4c-4693-b3b7-afe65193da82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:32.154613 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0f738f-5b4c-4693-b3b7-afe65193da82 None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:32.155116 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0f738f-5b4c-4693-b3b7-afe65193da82 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:32.155669 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0f738f-5b4c-4693-b3b7-afe65193da82 None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:32.156679 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0f738f-5b4c-4693-b3b7-afe65193da82 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:32.158618 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0f738f-5b4c-4693-b3b7-afe65193da82 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:32.232270 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b063eced-08d5-451d-9268-9df3cf353116 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:32.266077 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c6aa9f-b32f-4e5c-a01a-f6db5c776ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:32.266750 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c6aa9f-b32f-4e5c-a01a-f6db5c776ccb None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:32.267014 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c6aa9f-b32f-4e5c-a01a-f6db5c776ccb None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:32.267287 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c6aa9f-b32f-4e5c-a01a-f6db5c776ccb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:32.276113 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 115/217] 10.222.0.33 () {60 vars in 1132 bytes} [Fri Jul 15 19:14:32 2022] POST /identity/v3/users => generated 297 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:14:32.312872 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:32.357450 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c6aa9f-b32f-4e5c-a01a-f6db5c776ccb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:32.359050 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c6aa9f-b32f-4e5c-a01a-f6db5c776ccb None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:32.477229 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 103/218] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 15 19:14:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 19:14:32.482977 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0f738f-5b4c-4693-b3b7-afe65193da82 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:32.510545 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:32.535407 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84610d6-9b85-48b9-9831-376b8da37e8b None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:32.535703 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84610d6-9b85-48b9-9831-376b8da37e8b None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:32.535961 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84610d6-9b85-48b9-9831-376b8da37e8b None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:32.537490 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 104/219] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:14:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 19:14:32.542818 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813ad17f-ce69-4ac1-af36-d8011d7e0351 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:32.543224 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813ad17f-ce69-4ac1-af36-d8011d7e0351 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:32.543506 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813ad17f-ce69-4ac1-af36-d8011d7e0351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:32.563585 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-813ad17f-ce69-4ac1-af36-d8011d7e0351 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:14:32.564792 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-813ad17f-ce69-4ac1-af36-d8011d7e0351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:14:32.600276 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0f738f-5b4c-4693-b3b7-afe65193da82 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:32.640214 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d84cfb-7cc3-4c76-a58d-3940eaabd44b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:32.640214 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d84cfb-7cc3-4c76-a58d-3940eaabd44b None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:32.640480 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d84cfb-7cc3-4c76-a58d-3940eaabd44b None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:32.640726 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d84cfb-7cc3-4c76-a58d-3940eaabd44b None admin] PATH_INFO: `/v3/projects/12e6f4bc187942989f83aeb1902a3036` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:32.642454 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d84cfb-7cc3-4c76-a58d-3940eaabd44b None admin] RBAC: Authorizing `identity:get_project(project_id=12e6f4bc187942989f83aeb1902a3036)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:32.643942 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d84cfb-7cc3-4c76-a58d-3940eaabd44b None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:32.645821 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 116/220] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 15 19:14:32 2022] GET /identity/v3/projects/12e6f4bc187942989f83aeb1902a3036 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:14:32.653233 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d84cfb-7cc3-4c76-a58d-3940eaabd44b None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:32.658874 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:32.668599 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 105/221] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:14:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 19:14:32.694240 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813ad17f-ce69-4ac1-af36-d8011d7e0351 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:32.768854 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:32.787002 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d84cfb-7cc3-4c76-a58d-3940eaabd44b None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:32.821882 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813ad17f-ce69-4ac1-af36-d8011d7e0351 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:32.840130 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8aa418-b452-4a60-9bdf-cdc42c2115ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:32.842002 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8aa418-b452-4a60-9bdf-cdc42c2115ca None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:32.842488 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8aa418-b452-4a60-9bdf-cdc42c2115ca None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:32.844302 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8aa418-b452-4a60-9bdf-cdc42c2115ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:32.845337 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8aa418-b452-4a60-9bdf-cdc42c2115ca None admin] RBAC: Adding query filter params (role.id=28763c6e70094116b3a4b2895895fa58, scope.project.id=12e6f4bc187942989f83aeb1902a3036, user.id=605a9210f12a47ec8f5769bd86b7c636) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:32.846674 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8aa418-b452-4a60-9bdf-cdc42c2115ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:32.848331 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8aa418-b452-4a60-9bdf-cdc42c2115ca None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:32.863030 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 117/222] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 15 19:14:32 2022] GET /identity/v3/role_assignments?user.id=605a9210f12a47ec8f5769bd86b7c636&scope.project.id=12e6f4bc187942989f83aeb1902a3036&role.id=28763c6e70094116b3a4b2895895fa58 => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:32.876121 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f467851-126e-48bb-96b7-b158b9fec0df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:32.877255 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f467851-126e-48bb-96b7-b158b9fec0df None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:32.877753 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f467851-126e-48bb-96b7-b158b9fec0df None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:32.878232 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f467851-126e-48bb-96b7-b158b9fec0df None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:32.891126 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f467851-126e-48bb-96b7-b158b9fec0df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:14:32.899631 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-3f467851-126e-48bb-96b7-b158b9fec0df None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:14:32.899631 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:14:32.899631 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:14:32.899631 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:14:32.899631 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:14:32.899631 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:14:32.899631 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:14:32.899631 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:14:32.899631 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:14:32.899631 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:14:32.899631 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:14:32.899631 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:14:32.899631 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:14:32.899631 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:14:32.899631 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:14:32.899631 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:14:32.900587 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:14:32.900587 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:14:32.900587 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:14:32.900587 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:14:32.900587 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:14:32.900587 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:14:32.900587 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:14:32.900587 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:14:32.900587 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:14:32.900587 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:14:32.900587 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:14:32.900587 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:14:32.900587 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:14:32.900587 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:14:32.900587 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:14:32.900587 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:14:32.901380 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:14:32.901380 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:14:32.901380 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:14:32.901380 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:14:32.901380 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:14:32.901380 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:14:32.901380 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:14:32.901380 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:14:32.901380 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:14:32.901380 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:14:32.901380 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:14:32.901380 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:14:32.908149 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 106/223] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jul 15 19:14:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:14:32.909455 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8aa418-b452-4a60-9bdf-cdc42c2115ca None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:32.954802 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bb213b-43cd-4859-ad89-98b703c6b885 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:32.955185 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bb213b-43cd-4859-ad89-98b703c6b885 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:32.955467 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bb213b-43cd-4859-ad89-98b703c6b885 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:32.958009 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 107/224] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:14:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:14:32.965368 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a8be8b-ec69-49d4-8808-4b4b3c09fb10 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:32.965658 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a8be8b-ec69-49d4-8808-4b4b3c09fb10 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:32.965995 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a8be8b-ec69-49d4-8808-4b4b3c09fb10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:32.997060 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-83a8be8b-ec69-49d4-8808-4b4b3c09fb10 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:14:33.002012 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-83a8be8b-ec69-49d4-8808-4b4b3c09fb10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:14:33.058365 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:33.081537 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 108/225] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:14:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 19:14:33.106877 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a8be8b-ec69-49d4-8808-4b4b3c09fb10 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:33.117807 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8aa418-b452-4a60-9bdf-cdc42c2115ca None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:33.184543 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae046e95-bc2d-4f05-a54c-7aeace65acc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:33.185228 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae046e95-bc2d-4f05-a54c-7aeace65acc2 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:33.185514 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae046e95-bc2d-4f05-a54c-7aeace65acc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:33.185814 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae046e95-bc2d-4f05-a54c-7aeace65acc2 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:33.186590 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae046e95-bc2d-4f05-a54c-7aeace65acc2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:33.186902 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae046e95-bc2d-4f05-a54c-7aeace65acc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:33.189838 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae046e95-bc2d-4f05-a54c-7aeace65acc2 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:33.209384 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 118/226] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 15 19:14:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:33.216648 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae046e95-bc2d-4f05-a54c-7aeace65acc2 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:33.257247 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:33.310892 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a8be8b-ec69-49d4-8808-4b4b3c09fb10 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:33.380288 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33afe2a-7b93-4a75-8467-ccc851e7bd93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:33.381114 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33afe2a-7b93-4a75-8467-ccc851e7bd93 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:33.381394 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33afe2a-7b93-4a75-8467-ccc851e7bd93 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:33.381668 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33afe2a-7b93-4a75-8467-ccc851e7bd93 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:33.404839 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33afe2a-7b93-4a75-8467-ccc851e7bd93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:14:33.412949 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-f33afe2a-7b93-4a75-8467-ccc851e7bd93 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 19:14:33.412949 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:14:33.412949 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:14:33.412949 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:14:33.412949 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:14:33.412949 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:14:33.412949 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:14:33.412949 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:14:33.412949 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:14:33.412949 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:14:33.412949 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:14:33.412949 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:14:33.412949 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:14:33.412949 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:14:33.412949 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:14:33.412949 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:14:33.413876 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:14:33.413876 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:14:33.413876 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:14:33.413876 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:14:33.413876 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:14:33.413876 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:14:33.413876 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:14:33.413876 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:14:33.413876 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:14:33.413876 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:14:33.413876 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:14:33.413876 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:14:33.413876 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:14:33.413876 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:14:33.413876 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:14:33.413876 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:14:33.414723 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:14:33.414723 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:14:33.414723 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:14:33.414723 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:14:33.414723 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:14:33.414723 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:14:33.414723 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:14:33.414723 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:14:33.414723 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:14:33.414723 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:14:33.414723 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 19:14:33.414723 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:14:33.419028 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae046e95-bc2d-4f05-a54c-7aeace65acc2 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:33.427998 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 109/227] 10.222.0.33 () {56 vars in 1114 bytes} [Fri Jul 15 19:14:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:14:33.434102 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33afe2a-7b93-4a75-8467-ccc851e7bd93 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:33.478163 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2a6413-a123-4a09-9e89-fb421a3bfedc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:33.479040 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2a6413-a123-4a09-9e89-fb421a3bfedc None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:33.479317 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2a6413-a123-4a09-9e89-fb421a3bfedc None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:33.479588 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2a6413-a123-4a09-9e89-fb421a3bfedc None admin] PATH_INFO: `/v3/users/085b688691184e23a0ee3794af80b0e5` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:33.481534 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2a6413-a123-4a09-9e89-fb421a3bfedc None admin] RBAC: Authorizing `identity:get_user(user_id=085b688691184e23a0ee3794af80b0e5)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:33.483002 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2a6413-a123-4a09-9e89-fb421a3bfedc None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:33.485916 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 119/228] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 15 19:14:33 2022] GET /identity/v3/users/085b688691184e23a0ee3794af80b0e5 => generated 283 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:33.489998 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2a6413-a123-4a09-9e89-fb421a3bfedc None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:33.515686 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:33.604750 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33afe2a-7b93-4a75-8467-ccc851e7bd93 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:33.644444 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffb65f6-39ca-42ad-8aba-bcced5894bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:33.645391 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb65f6-39ca-42ad-8aba-bcced5894bc1 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:33.645848 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb65f6-39ca-42ad-8aba-bcced5894bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:33.646269 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb65f6-39ca-42ad-8aba-bcced5894bc1 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:33.648070 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffb65f6-39ca-42ad-8aba-bcced5894bc1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:33.648528 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffb65f6-39ca-42ad-8aba-bcced5894bc1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:33.650394 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffb65f6-39ca-42ad-8aba-bcced5894bc1 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:33.660524 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 110/229] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 15 19:14:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:33.667812 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffb65f6-39ca-42ad-8aba-bcced5894bc1 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:33.673666 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:33.693442 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2a6413-a123-4a09-9e89-fb421a3bfedc None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:33.730695 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccbf363-61fd-4d72-beec-97195ed74cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:33.731864 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccbf363-61fd-4d72-beec-97195ed74cc9 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:33.732283 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccbf363-61fd-4d72-beec-97195ed74cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:33.732694 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccbf363-61fd-4d72-beec-97195ed74cc9 None admin] PATH_INFO: `/v3/projects/12e6f4bc187942989f83aeb1902a3036` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:33.734115 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccbf363-61fd-4d72-beec-97195ed74cc9 None admin] RBAC: Authorizing `identity:get_project(project_id=12e6f4bc187942989f83aeb1902a3036)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:33.735953 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccbf363-61fd-4d72-beec-97195ed74cc9 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:33.737810 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 120/230] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 15 19:14:33 2022] GET /identity/v3/projects/12e6f4bc187942989f83aeb1902a3036 => generated 302 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:33.744494 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccbf363-61fd-4d72-beec-97195ed74cc9 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:33.859698 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccbf363-61fd-4d72-beec-97195ed74cc9 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:33.900294 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7404a2-c6d8-43f7-8aad-b31f74b9a424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:33.901204 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7404a2-c6d8-43f7-8aad-b31f74b9a424 None admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:33.901537 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7404a2-c6d8-43f7-8aad-b31f74b9a424 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:33.901848 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7404a2-c6d8-43f7-8aad-b31f74b9a424 None admin] PATH_INFO: `/v3/projects/12e6f4bc187942989f83aeb1902a3036/users/085b688691184e23a0ee3794af80b0e5/roles/4318f86ba9f74cd4a0bef1c311276657` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:33.903881 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7404a2-c6d8-43f7-8aad-b31f74b9a424 None admin] RBAC: Authorizing `identity:create_grant(project_id=12e6f4bc187942989f83aeb1902a3036, user_id=085b688691184e23a0ee3794af80b0e5, role_id=4318f86ba9f74cd4a0bef1c311276657)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:33.905323 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7404a2-c6d8-43f7-8aad-b31f74b9a424 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:33.939632 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffb65f6-39ca-42ad-8aba-bcced5894bc1 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:33.947411 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:33.975562 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a1d5f3-01f8-49a5-a865-825e28c455a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:33.976586 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a1d5f3-01f8-49a5-a865-825e28c455a5 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:33.976916 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a1d5f3-01f8-49a5-a865-825e28c455a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:33.977214 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a1d5f3-01f8-49a5-a865-825e28c455a5 None admin] PATH_INFO: `/v3/users/085b688691184e23a0ee3794af80b0e5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:33.982589 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a1d5f3-01f8-49a5-a865-825e28c455a5 None admin] RBAC: Authorizing `identity:get_user(user_id=085b688691184e23a0ee3794af80b0e5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:33.984566 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a1d5f3-01f8-49a5-a865-825e28c455a5 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:33.986229 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 111/231] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 15 19:14:33 2022] GET /identity/v3/users/085b688691184e23a0ee3794af80b0e5 => generated 283 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:33.991178 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a1d5f3-01f8-49a5-a865-825e28c455a5 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:34.012596 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 121/232] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jul 15 19:14:33 2022] PUT /identity/v3/projects/12e6f4bc187942989f83aeb1902a3036/users/085b688691184e23a0ee3794af80b0e5/roles/4318f86ba9f74cd4a0bef1c311276657 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:14:34.110396 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a1d5f3-01f8-49a5-a865-825e28c455a5 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:34.148386 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570b512c-c94b-4f48-a2b3-1a771058a68d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:34.150479 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b512c-c94b-4f48-a2b3-1a771058a68d None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:34.151003 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b512c-c94b-4f48-a2b3-1a771058a68d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:34.151477 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b512c-c94b-4f48-a2b3-1a771058a68d None admin] PATH_INFO: `/v3/projects/12e6f4bc187942989f83aeb1902a3036` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:34.152982 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570b512c-c94b-4f48-a2b3-1a771058a68d None admin] RBAC: Authorizing `identity:get_project(project_id=12e6f4bc187942989f83aeb1902a3036)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:34.154702 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570b512c-c94b-4f48-a2b3-1a771058a68d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:34.157084 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 112/233] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 15 19:14:33 2022] GET /identity/v3/projects/12e6f4bc187942989f83aeb1902a3036 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:14:34.163164 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7404a2-c6d8-43f7-8aad-b31f74b9a424 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:34.181006 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:34.290911 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7404a2-c6d8-43f7-8aad-b31f74b9a424 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:34.329593 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a26d7c-dc84-4507-9c34-6826bc5a9f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:34.330610 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a26d7c-dc84-4507-9c34-6826bc5a9f7b None admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:34.330894 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a26d7c-dc84-4507-9c34-6826bc5a9f7b None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:34.331168 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a26d7c-dc84-4507-9c34-6826bc5a9f7b None admin] PATH_INFO: `/v3/projects/12e6f4bc187942989f83aeb1902a3036/users/085b688691184e23a0ee3794af80b0e5/roles/e97d26621445466d9550261c608db9c5` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:34.333433 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a26d7c-dc84-4507-9c34-6826bc5a9f7b None admin] RBAC: Authorizing `identity:create_grant(project_id=12e6f4bc187942989f83aeb1902a3036, user_id=085b688691184e23a0ee3794af80b0e5, role_id=e97d26621445466d9550261c608db9c5)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:34.334920 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a26d7c-dc84-4507-9c34-6826bc5a9f7b None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:34.452173 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 122/234] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jul 15 19:14:34 2022] PUT /identity/v3/projects/12e6f4bc187942989f83aeb1902a3036/users/085b688691184e23a0ee3794af80b0e5/roles/e97d26621445466d9550261c608db9c5 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:14:34.925899 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc52ec77-4a9c-46f1-9987-e0386af60a38 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:34.926402 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc52ec77-4a9c-46f1-9987-e0386af60a38 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:34.926477 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc52ec77-4a9c-46f1-9987-e0386af60a38 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:34.928214 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 113/235] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:14:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:14:34.934124 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00261a5-78a4-42c6-a383-6ff7a64a992e None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:34.934555 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00261a5-78a4-42c6-a383-6ff7a64a992e None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:34.934781 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00261a5-78a4-42c6-a383-6ff7a64a992e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:34.958834 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba0fbd5-b323-4594-acb4-6f63048693b9 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:34.959468 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba0fbd5-b323-4594-acb4-6f63048693b9 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:34.959952 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba0fbd5-b323-4594-acb4-6f63048693b9 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:34.961682 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 114/236] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:14:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:14:34.962891 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-f00261a5-78a4-42c6-a383-6ff7a64a992e None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:14:34.964152 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-f00261a5-78a4-42c6-a383-6ff7a64a992e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:14:34.967508 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96cf39e-eff8-4c2e-b7e0-490266d461b9 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:34.968038 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96cf39e-eff8-4c2e-b7e0-490266d461b9 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:34.968541 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96cf39e-eff8-4c2e-b7e0-490266d461b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:34.998812 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-c96cf39e-eff8-4c2e-b7e0-490266d461b9 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:14:35.000505 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-c96cf39e-eff8-4c2e-b7e0-490266d461b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:14:35.066507 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 123/237] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:14:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:14:35.069848 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:35.092057 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00261a5-78a4-42c6-a383-6ff7a64a992e None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:35.100576 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:35.134463 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 115/238] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:14:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 19:14:35.166793 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc9c976-0e10-4f70-9710-342d5fe3b77d None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:35.167155 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc9c976-0e10-4f70-9710-342d5fe3b77d None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:35.167408 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc9c976-0e10-4f70-9710-342d5fe3b77d None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:35.169862 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 116/239] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:14:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:14:35.175631 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642f6a83-afe0-4086-bf44-20709fdba3ad None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:35.175923 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642f6a83-afe0-4086-bf44-20709fdba3ad None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:35.176201 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642f6a83-afe0-4086-bf44-20709fdba3ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:35.193801 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-642f6a83-afe0-4086-bf44-20709fdba3ad None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:14:35.195076 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-642f6a83-afe0-4086-bf44-20709fdba3ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:14:35.216519 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00261a5-78a4-42c6-a383-6ff7a64a992e None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:35.260867 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93760ee-e0fb-4e54-8e74-d3085518572a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:35.262820 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93760ee-e0fb-4e54-8e74-d3085518572a None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:35.263104 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93760ee-e0fb-4e54-8e74-d3085518572a None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:35.263378 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93760ee-e0fb-4e54-8e74-d3085518572a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:35.275342 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 117/240] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:14:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 19:14:35.275512 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93760ee-e0fb-4e54-8e74-d3085518572a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:14:35.282757 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642f6a83-afe0-4086-bf44-20709fdba3ad None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:35.283782 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-a93760ee-e0fb-4e54-8e74-d3085518572a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 19:14:35.283782 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:14:35.283782 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:14:35.283782 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:14:35.283782 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:14:35.283782 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:14:35.283782 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:14:35.283782 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:14:35.283782 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:14:35.283782 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:14:35.283782 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:14:35.283782 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:14:35.283782 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:14:35.283782 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:14:35.283782 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:14:35.283782 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:14:35.284662 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:14:35.284662 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:14:35.284662 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:14:35.284662 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:14:35.284662 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:14:35.284662 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:14:35.284662 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:14:35.284662 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:14:35.284662 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:14:35.284662 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:14:35.284662 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:14:35.284662 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:14:35.284662 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:14:35.284662 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:14:35.284662 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:14:35.284662 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:14:35.285466 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:14:35.285466 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:14:35.285466 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:14:35.285466 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:14:35.285466 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:14:35.285466 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:14:35.285466 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:14:35.285466 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:14:35.285466 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:14:35.285466 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:14:35.285466 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 19:14:35.285466 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:14:35.285466 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 124/241] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 15 19:14:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:14:35.291428 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93760ee-e0fb-4e54-8e74-d3085518572a None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:35.366465 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:35.408439 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93760ee-e0fb-4e54-8e74-d3085518572a None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:35.426211 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642f6a83-afe0-4086-bf44-20709fdba3ad None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:35.428422 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:35.446641 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c27b94-d1b8-4e7f-acf5-10bad881f33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:35.447401 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c27b94-d1b8-4e7f-acf5-10bad881f33d None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:35.447603 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c27b94-d1b8-4e7f-acf5-10bad881f33d None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:35.447882 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c27b94-d1b8-4e7f-acf5-10bad881f33d None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:35.448655 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c27b94-d1b8-4e7f-acf5-10bad881f33d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:35.449019 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c27b94-d1b8-4e7f-acf5-10bad881f33d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:35.450594 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c27b94-d1b8-4e7f-acf5-10bad881f33d None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:35.459809 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 125/242] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jul 15 19:14:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:14:35.461932 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c27b94-d1b8-4e7f-acf5-10bad881f33d None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:35.476160 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e22b62-d40b-48f9-bf70-5fa4b3b17454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:35.477678 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e22b62-d40b-48f9-bf70-5fa4b3b17454 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:35.478212 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e22b62-d40b-48f9-bf70-5fa4b3b17454 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:35.478924 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e22b62-d40b-48f9-bf70-5fa4b3b17454 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:35.481031 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e22b62-d40b-48f9-bf70-5fa4b3b17454 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:35.483126 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e22b62-d40b-48f9-bf70-5fa4b3b17454 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:35.485341 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 118/243] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:14:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:35.486364 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e22b62-d40b-48f9-bf70-5fa4b3b17454 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:35.588119 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c27b94-d1b8-4e7f-acf5-10bad881f33d None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:35.613533 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e22b62-d40b-48f9-bf70-5fa4b3b17454 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:35.620914 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:35.629895 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd2b5e9-b077-4161-961c-47a97d3b5cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:35.630869 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd2b5e9-b077-4161-961c-47a97d3b5cfd None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:35.631160 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd2b5e9-b077-4161-961c-47a97d3b5cfd None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:35.631439 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd2b5e9-b077-4161-961c-47a97d3b5cfd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:35.643714 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd2b5e9-b077-4161-961c-47a97d3b5cfd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:14:35.652386 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-ffd2b5e9-b077-4161-961c-47a97d3b5cfd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 19:14:35.652386 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:14:35.652386 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:14:35.652386 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:14:35.652386 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:14:35.652386 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:14:35.652386 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:14:35.652386 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:14:35.652386 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:14:35.652386 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:14:35.652386 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:14:35.652386 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:14:35.652386 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:14:35.652386 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:14:35.652386 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:14:35.652386 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:14:35.653297 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:14:35.653297 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:14:35.653297 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:14:35.653297 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:14:35.653297 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:14:35.653297 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:14:35.653297 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:14:35.653297 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:14:35.653297 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:14:35.653297 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:14:35.653297 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:14:35.653297 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:14:35.653297 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:14:35.653297 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:14:35.653297 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:14:35.653297 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:14:35.658870 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:14:35.658870 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:14:35.658870 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:14:35.658870 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:14:35.658870 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:14:35.658870 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:14:35.658870 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:14:35.658870 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:14:35.658870 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:14:35.658870 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:14:35.658870 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 19:14:35.658870 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:14:35.658870 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 126/244] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 15 19:14:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:14:35.658870 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd2b5e9-b077-4161-961c-47a97d3b5cfd None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:35.668425 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39cb5ba-6bfe-4ed9-b806-c0a1e42c316c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:35.670008 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39cb5ba-6bfe-4ed9-b806-c0a1e42c316c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:35.670615 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39cb5ba-6bfe-4ed9-b806-c0a1e42c316c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:35.671141 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39cb5ba-6bfe-4ed9-b806-c0a1e42c316c None admin] PATH_INFO: `/v3/users/3cbee5c59e184e51baa688dc2aeadf36` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:35.690268 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39cb5ba-6bfe-4ed9-b806-c0a1e42c316c None admin] RBAC: Authorizing `identity:get_user(user_id=3cbee5c59e184e51baa688dc2aeadf36)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:35.691751 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39cb5ba-6bfe-4ed9-b806-c0a1e42c316c None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:35.693444 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 119/245] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 15 19:14:35 2022] GET /identity/v3/users/3cbee5c59e184e51baa688dc2aeadf36 => generated 297 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:35.694249 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39cb5ba-6bfe-4ed9-b806-c0a1e42c316c None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:35.730490 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:35.776608 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd2b5e9-b077-4161-961c-47a97d3b5cfd None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:35.818013 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c73d3e-1122-4b77-b2a2-d435dba25a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:35.818966 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c73d3e-1122-4b77-b2a2-d435dba25a66 None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:35.819545 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c73d3e-1122-4b77-b2a2-d435dba25a66 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:35.820017 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c73d3e-1122-4b77-b2a2-d435dba25a66 None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:35.821551 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c73d3e-1122-4b77-b2a2-d435dba25a66 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:35.823347 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c73d3e-1122-4b77-b2a2-d435dba25a66 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:35.823790 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39cb5ba-6bfe-4ed9-b806-c0a1e42c316c None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:35.860526 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7921a0d-86f6-4f54-bec5-d3deeea9eb2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:35.861203 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7921a0d-86f6-4f54-bec5-d3deeea9eb2e None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:35.861485 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7921a0d-86f6-4f54-bec5-d3deeea9eb2e None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:35.861773 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7921a0d-86f6-4f54-bec5-d3deeea9eb2e None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:35.862532 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7921a0d-86f6-4f54-bec5-d3deeea9eb2e None admin] RBAC: Adding query filter params (name=member) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:35.862837 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7921a0d-86f6-4f54-bec5-d3deeea9eb2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:35.864232 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7921a0d-86f6-4f54-bec5-d3deeea9eb2e None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:35.873686 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 120/246] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jul 15 19:14:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:35.878541 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7921a0d-86f6-4f54-bec5-d3deeea9eb2e None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:35.983095 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:35.993166 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7921a0d-86f6-4f54-bec5-d3deeea9eb2e None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:36.026747 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b2636b-4358-4177-82bb-f4485479e962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:36.027404 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2636b-4358-4177-82bb-f4485479e962 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:36.027682 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2636b-4358-4177-82bb-f4485479e962 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:36.027952 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2636b-4358-4177-82bb-f4485479e962 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:36.093777 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b2636b-4358-4177-82bb-f4485479e962 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:36.095261 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b2636b-4358-4177-82bb-f4485479e962 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:36.193625 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 121/247] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 15 19:14:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 19:14:36.193625 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b2636b-4358-4177-82bb-f4485479e962 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:36.289490 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b2636b-4358-4177-82bb-f4485479e962 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:36.328844 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74374edf-8ab4-407d-8068-27ee01c4b7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:36.329725 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74374edf-8ab4-407d-8068-27ee01c4b7d9 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:36.330033 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74374edf-8ab4-407d-8068-27ee01c4b7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:36.330309 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74374edf-8ab4-407d-8068-27ee01c4b7d9 None admin] PATH_INFO: `/v3/users/605a9210f12a47ec8f5769bd86b7c636` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:36.331673 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74374edf-8ab4-407d-8068-27ee01c4b7d9 None admin] RBAC: Authorizing `identity:get_user(user_id=605a9210f12a47ec8f5769bd86b7c636)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:36.333129 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74374edf-8ab4-407d-8068-27ee01c4b7d9 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:36.335117 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 122/248] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 15 19:14:36 2022] GET /identity/v3/users/605a9210f12a47ec8f5769bd86b7c636 => generated 297 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:36.337831 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74374edf-8ab4-407d-8068-27ee01c4b7d9 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:36.436511 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74374edf-8ab4-407d-8068-27ee01c4b7d9 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:36.469569 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3569ea-1d93-4565-b47f-0a112c198788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:36.470397 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3569ea-1d93-4565-b47f-0a112c198788 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:36.470675 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3569ea-1d93-4565-b47f-0a112c198788 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:36.470945 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3569ea-1d93-4565-b47f-0a112c198788 None admin] PATH_INFO: `/v3/projects/12e6f4bc187942989f83aeb1902a3036` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:36.472120 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3569ea-1d93-4565-b47f-0a112c198788 None admin] RBAC: Authorizing `identity:get_project(project_id=12e6f4bc187942989f83aeb1902a3036)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:36.473549 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3569ea-1d93-4565-b47f-0a112c198788 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:36.475286 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 123/249] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 15 19:14:36 2022] GET /identity/v3/projects/12e6f4bc187942989f83aeb1902a3036 => generated 302 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:36.476048 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3569ea-1d93-4565-b47f-0a112c198788 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:36.573594 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3569ea-1d93-4565-b47f-0a112c198788 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:36.582183 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 127/250] 10.222.0.33 () {60 vars in 1132 bytes} [Fri Jul 15 19:14:35 2022] POST /identity/v3/users => generated 293 bytes in 928 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:14:36.583486 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1694ee-e713-49a5-8b97-47b2935b222d None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:36.584018 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1694ee-e713-49a5-8b97-47b2935b222d None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:36.584123 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1694ee-e713-49a5-8b97-47b2935b222d None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:36.585507 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 128/251] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:14:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:14:36.590940 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1694ee-e713-49a5-8b97-47b2935b222d None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:36.613450 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:36.624707 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afee9a54-38a2-42c6-861b-7eec51ee6c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:36.626059 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afee9a54-38a2-42c6-861b-7eec51ee6c40 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:36.626521 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afee9a54-38a2-42c6-861b-7eec51ee6c40 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:36.627254 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afee9a54-38a2-42c6-861b-7eec51ee6c40 None admin] PATH_INFO: `/v3/projects/12e6f4bc187942989f83aeb1902a3036` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:36.628996 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afee9a54-38a2-42c6-861b-7eec51ee6c40 None admin] RBAC: Authorizing `identity:get_project(project_id=12e6f4bc187942989f83aeb1902a3036)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:36.630837 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afee9a54-38a2-42c6-861b-7eec51ee6c40 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:36.633785 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 124/252] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 15 19:14:36 2022] GET /identity/v3/projects/12e6f4bc187942989f83aeb1902a3036 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:36.636065 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83729237-2a0b-4ded-954a-885c99c41506 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:36.636674 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83729237-2a0b-4ded-954a-885c99c41506 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:36.637157 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83729237-2a0b-4ded-954a-885c99c41506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:36.649348 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:36.660281 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-83729237-2a0b-4ded-954a-885c99c41506 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:14:36.661661 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-83729237-2a0b-4ded-954a-885c99c41506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:14:36.727570 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1694ee-e713-49a5-8b97-47b2935b222d None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:36.737336 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:36.754017 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 125/253] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:14:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:14:36.754613 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83729237-2a0b-4ded-954a-885c99c41506 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:36.779887 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4588124b-12a5-40e5-b0e1-618c1869f379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:36.780555 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4588124b-12a5-40e5-b0e1-618c1869f379 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:36.780940 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4588124b-12a5-40e5-b0e1-618c1869f379 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:36.781219 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4588124b-12a5-40e5-b0e1-618c1869f379 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:36.782850 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4588124b-12a5-40e5-b0e1-618c1869f379 None admin] RBAC: Adding query filter params (role.id=2f62da1ebb2b443eabf21e910a215048, scope.project.id=12e6f4bc187942989f83aeb1902a3036, user.id=3cbee5c59e184e51baa688dc2aeadf36) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:36.783328 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4588124b-12a5-40e5-b0e1-618c1869f379 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:36.784933 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4588124b-12a5-40e5-b0e1-618c1869f379 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:36.791932 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 129/254] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 15 19:14:36 2022] GET /identity/v3/role_assignments?user.id=3cbee5c59e184e51baa688dc2aeadf36&scope.project.id=12e6f4bc187942989f83aeb1902a3036&role.id=2f62da1ebb2b443eabf21e910a215048 => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:36.793064 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4588124b-12a5-40e5-b0e1-618c1869f379 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:36.803271 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:36.902328 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83729237-2a0b-4ded-954a-885c99c41506 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:36.931815 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4588124b-12a5-40e5-b0e1-618c1869f379 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:36.938076 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:36.940286 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:36.955539 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72963d18-6845-4fde-a8ac-e68db5ec6ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:36.957053 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72963d18-6845-4fde-a8ac-e68db5ec6ffd None admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:36.957984 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72963d18-6845-4fde-a8ac-e68db5ec6ffd None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:36.958505 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72963d18-6845-4fde-a8ac-e68db5ec6ffd None admin] PATH_INFO: `/v3/projects/12e6f4bc187942989f83aeb1902a3036/users/605a9210f12a47ec8f5769bd86b7c636/roles/28763c6e70094116b3a4b2895895fa58` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:36.962127 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72963d18-6845-4fde-a8ac-e68db5ec6ffd None admin] RBAC: Authorizing `identity:create_grant(project_id=12e6f4bc187942989f83aeb1902a3036, user_id=605a9210f12a47ec8f5769bd86b7c636, role_id=28763c6e70094116b3a4b2895895fa58)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:36.964561 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72963d18-6845-4fde-a8ac-e68db5ec6ffd None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:36.972128 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d206e5f-5e08-4d03-bb92-c1e0412bf8a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:36.973037 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d206e5f-5e08-4d03-bb92-c1e0412bf8a8 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:36.973231 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d206e5f-5e08-4d03-bb92-c1e0412bf8a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:36.973487 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d206e5f-5e08-4d03-bb92-c1e0412bf8a8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:36.984714 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d206e5f-5e08-4d03-bb92-c1e0412bf8a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:14:36.992911 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-3d206e5f-5e08-4d03-bb92-c1e0412bf8a8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:14:36.992911 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:14:36.992911 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:14:36.992911 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:14:36.992911 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:14:36.992911 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:14:36.992911 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:14:36.992911 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:14:36.992911 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:14:36.992911 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:14:36.992911 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:14:36.992911 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:14:36.992911 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:14:36.992911 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:14:36.992911 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:14:36.992911 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:14:36.996522 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:14:36.996522 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:14:36.996522 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:14:36.996522 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:14:36.996522 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:14:36.996522 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:14:36.996522 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:14:36.996522 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:14:36.996522 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:14:36.996522 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:14:36.996522 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:14:36.996522 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:14:36.996522 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:14:36.996522 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:14:36.996522 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:14:36.996522 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:14:36.997329 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:14:36.997329 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:14:36.997329 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:14:36.997329 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:14:36.997329 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:14:36.997329 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:14:36.997329 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:14:36.997329 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:14:36.997329 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:14:36.997329 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:14:36.997329 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:14:36.997329 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:14:36.997329 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 130/255] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jul 15 19:14:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:14:36.997329 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9699dbc1-c2a2-491e-89d9-c20bb517f95d None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:36.998099 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9699dbc1-c2a2-491e-89d9-c20bb517f95d None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:36.998099 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9699dbc1-c2a2-491e-89d9-c20bb517f95d None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:36.998099 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 131/256] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:14:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:14:37.001272 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9699dbc1-c2a2-491e-89d9-c20bb517f95d None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:37.070749 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 126/257] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jul 15 19:14:36 2022] PUT /identity/v3/projects/12e6f4bc187942989f83aeb1902a3036/users/605a9210f12a47ec8f5769bd86b7c636/roles/28763c6e70094116b3a4b2895895fa58 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:14:37.072802 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb3fe8-533c-4aab-9915-a0839a0b1e5a None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:37.073275 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb3fe8-533c-4aab-9915-a0839a0b1e5a None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:37.073755 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb3fe8-533c-4aab-9915-a0839a0b1e5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:37.095725 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-f1bb3fe8-533c-4aab-9915-a0839a0b1e5a None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:14:37.097698 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-f1bb3fe8-533c-4aab-9915-a0839a0b1e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:14:37.129896 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9699dbc1-c2a2-491e-89d9-c20bb517f95d None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:37.202031 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5f9073-e0b7-4227-ad4e-c1f1936bafdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:37.211137 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5f9073-e0b7-4227-ad4e-c1f1936bafdd None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:37.211591 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5f9073-e0b7-4227-ad4e-c1f1936bafdd None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:37.211980 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5f9073-e0b7-4227-ad4e-c1f1936bafdd None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:37.212816 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5f9073-e0b7-4227-ad4e-c1f1936bafdd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:37.213124 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5f9073-e0b7-4227-ad4e-c1f1936bafdd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:37.214596 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5f9073-e0b7-4227-ad4e-c1f1936bafdd None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:37.224998 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 132/258] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 15 19:14:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:37.232702 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5f9073-e0b7-4227-ad4e-c1f1936bafdd None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:37.237581 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 127/259] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:14:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:14:37.278825 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:37.318160 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bb3fe8-533c-4aab-9915-a0839a0b1e5a None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:37.365256 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5f9073-e0b7-4227-ad4e-c1f1936bafdd None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:37.406506 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24efbf08-311d-4a91-a4f3-a8e3dfaeb566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:37.407477 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24efbf08-311d-4a91-a4f3-a8e3dfaeb566 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:37.407782 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24efbf08-311d-4a91-a4f3-a8e3dfaeb566 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:37.408121 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24efbf08-311d-4a91-a4f3-a8e3dfaeb566 None admin] PATH_INFO: `/v3/users/085b688691184e23a0ee3794af80b0e5` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:37.409319 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24efbf08-311d-4a91-a4f3-a8e3dfaeb566 None admin] RBAC: Authorizing `identity:get_user(user_id=085b688691184e23a0ee3794af80b0e5)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:37.410804 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24efbf08-311d-4a91-a4f3-a8e3dfaeb566 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:37.412860 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 133/260] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 15 19:14:37 2022] GET /identity/v3/users/085b688691184e23a0ee3794af80b0e5 => generated 283 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:14:37.419224 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24efbf08-311d-4a91-a4f3-a8e3dfaeb566 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:37.456396 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bb3fe8-533c-4aab-9915-a0839a0b1e5a None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:37.476166 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:37.524078 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0fb338-459b-463d-8de7-a67ce0db710c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:37.525166 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0fb338-459b-463d-8de7-a67ce0db710c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:37.525643 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0fb338-459b-463d-8de7-a67ce0db710c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:37.526072 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0fb338-459b-463d-8de7-a67ce0db710c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:37.546194 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0fb338-459b-463d-8de7-a67ce0db710c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:14:37.560525 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-db0fb338-459b-463d-8de7-a67ce0db710c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 19:14:37.560525 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:14:37.560525 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:14:37.560525 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:14:37.560525 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:14:37.560525 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:14:37.560525 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:14:37.560525 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:14:37.560525 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:14:37.560525 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:14:37.560525 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:14:37.560525 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:14:37.560525 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:14:37.560525 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:14:37.560525 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:14:37.560525 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:14:37.561657 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:14:37.561657 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:14:37.561657 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:14:37.561657 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:14:37.561657 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:14:37.561657 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:14:37.561657 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:14:37.561657 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:14:37.561657 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:14:37.561657 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:14:37.561657 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:14:37.561657 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:14:37.561657 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:14:37.561657 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:14:37.561657 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:14:37.561657 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:14:37.562539 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:14:37.562539 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:14:37.562539 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:14:37.562539 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:14:37.562539 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:14:37.562539 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:14:37.562539 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:14:37.562539 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:14:37.562539 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:14:37.562539 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:14:37.562539 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 19:14:37.562539 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:14:37.564570 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:37.567371 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 128/261] 10.222.0.33 () {56 vars in 1114 bytes} [Fri Jul 15 19:14:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:14:37.573060 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24efbf08-311d-4a91-a4f3-a8e3dfaeb566 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:37.574145 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0fb338-459b-463d-8de7-a67ce0db710c None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:37.640530 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2a176b-43ce-4b70-b934-eb2762d8c425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:37.641976 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2a176b-43ce-4b70-b934-eb2762d8c425 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:37.642487 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2a176b-43ce-4b70-b934-eb2762d8c425 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:37.643668 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2a176b-43ce-4b70-b934-eb2762d8c425 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:37.822986 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2a176b-43ce-4b70-b934-eb2762d8c425 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:37.824462 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2a176b-43ce-4b70-b934-eb2762d8c425 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:37.843394 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0fb338-459b-463d-8de7-a67ce0db710c None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:37.919114 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a86cc7-085c-4ae0-8b48-64d0bc2cf2ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:37.919791 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a86cc7-085c-4ae0-8b48-64d0bc2cf2ad None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:37.920070 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a86cc7-085c-4ae0-8b48-64d0bc2cf2ad None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:37.920339 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a86cc7-085c-4ae0-8b48-64d0bc2cf2ad None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:37.921100 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a86cc7-085c-4ae0-8b48-64d0bc2cf2ad None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:37.921401 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a86cc7-085c-4ae0-8b48-64d0bc2cf2ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:37.922814 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a86cc7-085c-4ae0-8b48-64d0bc2cf2ad None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:37.943731 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 129/262] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 15 19:14:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:37.946414 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a86cc7-085c-4ae0-8b48-64d0bc2cf2ad None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:37.990220 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:38.003460 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 134/263] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 15 19:14:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 585 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 15 19:14:38.009531 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2a176b-43ce-4b70-b934-eb2762d8c425 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:38.038804 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:38.213660 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2a176b-43ce-4b70-b934-eb2762d8c425 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:38.235727 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a86cc7-085c-4ae0-8b48-64d0bc2cf2ad None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:38.273861 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfff426-3a1c-4d9c-8517-f730db68cc66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:38.274745 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfff426-3a1c-4d9c-8517-f730db68cc66 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:38.275025 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfff426-3a1c-4d9c-8517-f730db68cc66 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:38.275297 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfff426-3a1c-4d9c-8517-f730db68cc66 None admin] PATH_INFO: `/v3/projects/12e6f4bc187942989f83aeb1902a3036` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:38.279667 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfff426-3a1c-4d9c-8517-f730db68cc66 None admin] RBAC: Authorizing `identity:get_project(project_id=12e6f4bc187942989f83aeb1902a3036)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:38.281134 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfff426-3a1c-4d9c-8517-f730db68cc66 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:38.283191 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 135/264] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 15 19:14:38 2022] GET /identity/v3/projects/12e6f4bc187942989f83aeb1902a3036 => generated 302 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:14:38.289034 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfff426-3a1c-4d9c-8517-f730db68cc66 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:38.353347 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa19cf1d-f084-4545-907d-213dd32edde6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:38.354272 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa19cf1d-f084-4545-907d-213dd32edde6 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:38.354556 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa19cf1d-f084-4545-907d-213dd32edde6 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:38.354836 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa19cf1d-f084-4545-907d-213dd32edde6 None admin] PATH_INFO: `/v3/users/085b688691184e23a0ee3794af80b0e5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:38.360592 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa19cf1d-f084-4545-907d-213dd32edde6 None admin] RBAC: Authorizing `identity:get_user(user_id=085b688691184e23a0ee3794af80b0e5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:38.362109 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa19cf1d-f084-4545-907d-213dd32edde6 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:38.364229 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 130/265] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 15 19:14:37 2022] GET /identity/v3/users/085b688691184e23a0ee3794af80b0e5 => generated 283 bytes in 418 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:38.375435 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa19cf1d-f084-4545-907d-213dd32edde6 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:38.397234 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:38.475922 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfff426-3a1c-4d9c-8517-f730db68cc66 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:38.537258 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa19cf1d-f084-4545-907d-213dd32edde6 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:38.539790 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d86093e-a218-4ea6-9a1c-be6eb7ad8872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:38.540143 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:38.541100 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d86093e-a218-4ea6-9a1c-be6eb7ad8872 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:38.541381 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d86093e-a218-4ea6-9a1c-be6eb7ad8872 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:38.541653 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d86093e-a218-4ea6-9a1c-be6eb7ad8872 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:38.542527 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d86093e-a218-4ea6-9a1c-be6eb7ad8872 None admin] RBAC: Adding query filter params (role.id=4318f86ba9f74cd4a0bef1c311276657, scope.project.id=12e6f4bc187942989f83aeb1902a3036, user.id=085b688691184e23a0ee3794af80b0e5) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:38.542839 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d86093e-a218-4ea6-9a1c-be6eb7ad8872 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:38.544367 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d86093e-a218-4ea6-9a1c-be6eb7ad8872 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:38.559966 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 136/266] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 15 19:14:38 2022] GET /identity/v3/role_assignments?user.id=085b688691184e23a0ee3794af80b0e5&scope.project.id=12e6f4bc187942989f83aeb1902a3036&role.id=4318f86ba9f74cd4a0bef1c311276657 => generated 608 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:38.584306 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f7f1ab-c7d4-4800-b16a-719ac8f7fee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:38.585356 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f7f1ab-c7d4-4800-b16a-719ac8f7fee3 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:38.585909 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f7f1ab-c7d4-4800-b16a-719ac8f7fee3 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:38.586406 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f7f1ab-c7d4-4800-b16a-719ac8f7fee3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:38.682578 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f7f1ab-c7d4-4800-b16a-719ac8f7fee3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:38.684476 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f7f1ab-c7d4-4800-b16a-719ac8f7fee3 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:38.829856 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 131/267] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 15 19:14:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 19:14:38.835493 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d86093e-a218-4ea6-9a1c-be6eb7ad8872 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:38.970785 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d86093e-a218-4ea6-9a1c-be6eb7ad8872 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:38.999040 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:39.010448 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c96ff9-3e5a-407d-bbca-950840a04ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:39.011416 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c96ff9-3e5a-407d-bbca-950840a04ed8 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:39.011704 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c96ff9-3e5a-407d-bbca-950840a04ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:39.011983 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c96ff9-3e5a-407d-bbca-950840a04ed8 None admin] PATH_INFO: `/v3/projects/12e6f4bc187942989f83aeb1902a3036` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:39.013327 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c96ff9-3e5a-407d-bbca-950840a04ed8 None admin] RBAC: Authorizing `identity:get_project(project_id=12e6f4bc187942989f83aeb1902a3036)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:39.014883 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c96ff9-3e5a-407d-bbca-950840a04ed8 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:39.017019 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 137/268] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 15 19:14:38 2022] GET /identity/v3/projects/12e6f4bc187942989f83aeb1902a3036 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:39.024177 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f7f1ab-c7d4-4800-b16a-719ac8f7fee3 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:39.031501 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:39.154732 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f7f1ab-c7d4-4800-b16a-719ac8f7fee3 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:39.202210 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e344b9bd-e8e4-445b-a8af-c56e0bf1301a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:39.202951 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e344b9bd-e8e4-445b-a8af-c56e0bf1301a None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:39.203238 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e344b9bd-e8e4-445b-a8af-c56e0bf1301a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:39.203520 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e344b9bd-e8e4-445b-a8af-c56e0bf1301a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:39.204376 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e344b9bd-e8e4-445b-a8af-c56e0bf1301a None admin] RBAC: Adding query filter params (role.id=e97d26621445466d9550261c608db9c5, scope.project.id=12e6f4bc187942989f83aeb1902a3036, user.id=085b688691184e23a0ee3794af80b0e5) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:39.204692 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e344b9bd-e8e4-445b-a8af-c56e0bf1301a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:39.206185 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e344b9bd-e8e4-445b-a8af-c56e0bf1301a None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:39.213362 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 132/269] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 15 19:14:39 2022] GET /identity/v3/role_assignments?user.id=085b688691184e23a0ee3794af80b0e5&scope.project.id=12e6f4bc187942989f83aeb1902a3036&role.id=e97d26621445466d9550261c608db9c5 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:39.664379 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecfe6c5-7775-4021-bd8d-19c3930713fc None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:39.664753 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecfe6c5-7775-4021-bd8d-19c3930713fc None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:39.664961 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecfe6c5-7775-4021-bd8d-19c3930713fc None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:39.666486 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 138/270] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:14:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:14:39.672284 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd175a-46e0-4470-a589-bfd292aa05fb None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:39.672727 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd175a-46e0-4470-a589-bfd292aa05fb None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:39.672900 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd175a-46e0-4470-a589-bfd292aa05fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:39.696790 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-20dd175a-46e0-4470-a589-bfd292aa05fb None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:14:39.698342 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-20dd175a-46e0-4470-a589-bfd292aa05fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:14:39.723621 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff84276-1dbf-4f32-a9d8-10ae47ee6697 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:39.723621 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff84276-1dbf-4f32-a9d8-10ae47ee6697 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:39.723621 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff84276-1dbf-4f32-a9d8-10ae47ee6697 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:39.724692 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 139/271] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:14:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:14:39.731926 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1419dba1-609c-4d2c-a0d7-097e10dcc221 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:39.732542 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1419dba1-609c-4d2c-a0d7-097e10dcc221 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:39.733140 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1419dba1-609c-4d2c-a0d7-097e10dcc221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:39.752535 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-1419dba1-609c-4d2c-a0d7-097e10dcc221 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:14:39.754470 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-1419dba1-609c-4d2c-a0d7-097e10dcc221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:14:39.787458 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:39.794085 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 133/272] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:14:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:14:39.822437 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dd175a-46e0-4470-a589-bfd292aa05fb None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:39.856663 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 140/273] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:14:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 19:14:39.858031 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea06579-0179-42e7-a6ff-c8d9e839921e None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:39.858317 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea06579-0179-42e7-a6ff-c8d9e839921e None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:39.858650 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea06579-0179-42e7-a6ff-c8d9e839921e None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:39.860128 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 141/274] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:14:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:14:39.866124 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffd9529-c196-465f-9006-9c694958b1f4 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:39.866665 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffd9529-c196-465f-9006-9c694958b1f4 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:39.867202 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffd9529-c196-465f-9006-9c694958b1f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:39.885394 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-2ffd9529-c196-465f-9006-9c694958b1f4 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:14:39.886627 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-2ffd9529-c196-465f-9006-9c694958b1f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:14:39.932838 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dd175a-46e0-4470-a589-bfd292aa05fb None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:39.952354 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 142/275] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:14:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 19:14:39.953095 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffd9529-c196-465f-9006-9c694958b1f4 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:39.968681 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d100483-8ed3-4b4a-ae0f-a37bf48359d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:39.969477 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d100483-8ed3-4b4a-ae0f-a37bf48359d9 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:39.969758 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d100483-8ed3-4b4a-ae0f-a37bf48359d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:39.970057 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d100483-8ed3-4b4a-ae0f-a37bf48359d9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:39.980690 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d100483-8ed3-4b4a-ae0f-a37bf48359d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:14:39.988411 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-4d100483-8ed3-4b4a-ae0f-a37bf48359d9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 19:14:39.988411 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:14:39.988411 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:14:39.988411 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:14:39.988411 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:14:39.988411 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:14:39.988411 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:14:39.988411 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:14:39.988411 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:14:39.988411 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:14:39.988411 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:14:39.988411 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:14:39.988411 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:14:39.988411 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:14:39.988411 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:14:39.988411 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:14:39.989321 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:14:39.989321 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:14:39.989321 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:14:39.989321 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:14:39.989321 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:14:39.989321 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:14:39.989321 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:14:39.989321 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:14:39.989321 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:14:39.989321 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:14:39.989321 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:14:39.989321 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:14:39.989321 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:14:39.989321 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:14:39.989321 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:14:39.989321 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:14:39.990145 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:14:39.990145 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:14:39.990145 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:14:39.990145 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:14:39.990145 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:14:39.990145 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:14:39.990145 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:14:39.990145 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:14:39.990145 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:14:39.990145 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:14:39.990145 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 19:14:39.990145 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:14:39.990145 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 134/276] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 15 19:14:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:14:39.990931 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de72aa6f-b1c2-40eb-af18-f328719e5374 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:39.991115 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de72aa6f-b1c2-40eb-af18-f328719e5374 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:39.991393 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de72aa6f-b1c2-40eb-af18-f328719e5374 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:39.992769 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 135/277] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:14:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:14:39.993466 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de72aa6f-b1c2-40eb-af18-f328719e5374 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:40.012593 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:40.053642 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:40.055526 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffd9529-c196-465f-9006-9c694958b1f4 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:40.089824 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acf745c-5f16-448f-8984-ccb5e88da84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:40.090620 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acf745c-5f16-448f-8984-ccb5e88da84a None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:40.090904 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acf745c-5f16-448f-8984-ccb5e88da84a None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:40.091176 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acf745c-5f16-448f-8984-ccb5e88da84a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:40.095161 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de72aa6f-b1c2-40eb-af18-f328719e5374 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:40.101782 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acf745c-5f16-448f-8984-ccb5e88da84a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:14:40.109452 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-5acf745c-5f16-448f-8984-ccb5e88da84a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 19:14:40.109452 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:14:40.109452 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:14:40.109452 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:14:40.109452 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:14:40.109452 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:14:40.109452 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:14:40.109452 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:14:40.109452 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:14:40.109452 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:14:40.109452 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:14:40.109452 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:14:40.109452 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:14:40.109452 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:14:40.109452 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:14:40.109452 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:14:40.110529 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:14:40.110529 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:14:40.110529 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:14:40.110529 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:14:40.110529 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:14:40.110529 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:14:40.110529 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:14:40.110529 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:14:40.110529 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:14:40.110529 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:14:40.110529 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:14:40.110529 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:14:40.110529 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:14:40.110529 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:14:40.110529 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:14:40.110529 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:14:40.111623 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:14:40.111623 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:14:40.111623 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:14:40.111623 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:14:40.111623 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:14:40.111623 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:14:40.111623 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:14:40.111623 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:14:40.111623 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:14:40.111623 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:14:40.111623 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 19:14:40.111623 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:14:40.111623 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 143/278] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 15 19:14:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:14:40.111623 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acf745c-5f16-448f-8984-ccb5e88da84a None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:40.132575 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e877bb-47fd-4465-a5d2-a9de7dde8d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:40.133935 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e877bb-47fd-4465-a5d2-a9de7dde8d88 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:40.134216 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e877bb-47fd-4465-a5d2-a9de7dde8d88 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:40.134577 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e877bb-47fd-4465-a5d2-a9de7dde8d88 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:40.135740 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e877bb-47fd-4465-a5d2-a9de7dde8d88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:40.137173 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e877bb-47fd-4465-a5d2-a9de7dde8d88 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:40.138819 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 136/279] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:14:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:40.138819 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8a5916-3ddf-4d9a-adc9-4632f12e2573 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:40.138819 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8a5916-3ddf-4d9a-adc9-4632f12e2573 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:40.138819 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8a5916-3ddf-4d9a-adc9-4632f12e2573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:40.157212 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-6b8a5916-3ddf-4d9a-adc9-4632f12e2573 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:14:40.158417 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-6b8a5916-3ddf-4d9a-adc9-4632f12e2573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:14:40.216513 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acf745c-5f16-448f-8984-ccb5e88da84a None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:40.225338 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 137/280] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:14:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:14:40.226210 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8a5916-3ddf-4d9a-adc9-4632f12e2573 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:40.238999 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:40.251730 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12b3f7b-d250-43ef-b5e5-a1f55d64ecd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:40.252395 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b3f7b-d250-43ef-b5e5-a1f55d64ecd0 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:40.252675 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b3f7b-d250-43ef-b5e5-a1f55d64ecd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:40.252949 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b3f7b-d250-43ef-b5e5-a1f55d64ecd0 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:40.253699 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12b3f7b-d250-43ef-b5e5-a1f55d64ecd0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:40.254043 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12b3f7b-d250-43ef-b5e5-a1f55d64ecd0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:40.255428 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12b3f7b-d250-43ef-b5e5-a1f55d64ecd0 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:40.264585 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 144/281] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jul 15 19:14:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:40.265359 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12b3f7b-d250-43ef-b5e5-a1f55d64ecd0 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:40.286527 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:40.329832 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8a5916-3ddf-4d9a-adc9-4632f12e2573 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:40.362472 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12b3f7b-d250-43ef-b5e5-a1f55d64ecd0 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:40.362696 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39429ee0-ef7e-426b-a62a-39fa0d143a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:40.363327 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39429ee0-ef7e-426b-a62a-39fa0d143a49 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:40.363682 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39429ee0-ef7e-426b-a62a-39fa0d143a49 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:40.363876 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39429ee0-ef7e-426b-a62a-39fa0d143a49 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:40.365305 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39429ee0-ef7e-426b-a62a-39fa0d143a49 None admin] RBAC: Adding query filter params (name=member) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:40.365305 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39429ee0-ef7e-426b-a62a-39fa0d143a49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:40.366475 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39429ee0-ef7e-426b-a62a-39fa0d143a49 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:40.375199 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 138/282] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jul 15 19:14:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:40.375199 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39429ee0-ef7e-426b-a62a-39fa0d143a49 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:40.403226 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f19a2b5-6f3a-4b33-8cc3-cf77f00c467d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:40.403893 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f19a2b5-6f3a-4b33-8cc3-cf77f00c467d None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:40.404173 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f19a2b5-6f3a-4b33-8cc3-cf77f00c467d None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:40.404455 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f19a2b5-6f3a-4b33-8cc3-cf77f00c467d None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:40.405158 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f19a2b5-6f3a-4b33-8cc3-cf77f00c467d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:40.406630 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f19a2b5-6f3a-4b33-8cc3-cf77f00c467d None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:40.473790 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39429ee0-ef7e-426b-a62a-39fa0d143a49 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:40.490027 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:40.506542 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f842367-a4c1-4bd3-8716-3c353a9300b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:40.507329 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f842367-a4c1-4bd3-8716-3c353a9300b4 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:40.507609 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f842367-a4c1-4bd3-8716-3c353a9300b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:40.508527 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f842367-a4c1-4bd3-8716-3c353a9300b4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:40.518659 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f842367-a4c1-4bd3-8716-3c353a9300b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:14:40.526488 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-4f842367-a4c1-4bd3-8716-3c353a9300b4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 19:14:40.526488 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:14:40.526488 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:14:40.526488 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:14:40.526488 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:14:40.526488 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:14:40.526488 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:14:40.526488 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:14:40.526488 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:14:40.526488 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:14:40.526488 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:14:40.526488 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:14:40.526488 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:14:40.526488 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:14:40.526488 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:14:40.526488 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:14:40.527538 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:14:40.527538 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:14:40.527538 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:14:40.527538 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:14:40.527538 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:14:40.527538 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:14:40.527538 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:14:40.527538 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:14:40.527538 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:14:40.527538 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:14:40.527538 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:14:40.527538 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:14:40.527538 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:14:40.527538 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:14:40.527538 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:14:40.527538 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:14:40.528364 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:14:40.528364 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:14:40.528364 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:14:40.528364 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:14:40.528364 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:14:40.528364 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:14:40.528364 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:14:40.528364 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:14:40.528364 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:14:40.528364 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:14:40.528364 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 19:14:40.528364 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:14:40.528364 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 139/283] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 15 19:14:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:14:40.529088 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f842367-a4c1-4bd3-8716-3c353a9300b4 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:40.539173 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 145/284] 10.222.0.33 () {60 vars in 1132 bytes} [Fri Jul 15 19:14:40 2022] POST /identity/v3/users => generated 293 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:14:40.540105 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f19a2b5-6f3a-4b33-8cc3-cf77f00c467d None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:40.579517 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:40.646195 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f842367-a4c1-4bd3-8716-3c353a9300b4 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:40.663299 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f19a2b5-6f3a-4b33-8cc3-cf77f00c467d None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:40.686455 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427e013a-f951-4467-b992-c044d69d16ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:40.687794 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e013a-f951-4467-b992-c044d69d16ec None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:40.688073 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e013a-f951-4467-b992-c044d69d16ec None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:40.688496 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e013a-f951-4467-b992-c044d69d16ec None admin] PATH_INFO: `/v3/users/3cbee5c59e184e51baa688dc2aeadf36` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:40.689877 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427e013a-f951-4467-b992-c044d69d16ec None admin] RBAC: Authorizing `identity:get_user(user_id=3cbee5c59e184e51baa688dc2aeadf36)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:40.691813 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427e013a-f951-4467-b992-c044d69d16ec None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:40.694208 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 140/285] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 15 19:14:40 2022] GET /identity/v3/users/3cbee5c59e184e51baa688dc2aeadf36 => generated 297 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:40.695420 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427e013a-f951-4467-b992-c044d69d16ec None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:40.703772 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dbcea1-9ca3-43d3-9525-65b04ea163ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:40.705174 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dbcea1-9ca3-43d3-9525-65b04ea163ec None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:40.705454 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dbcea1-9ca3-43d3-9525-65b04ea163ec None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:40.705727 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dbcea1-9ca3-43d3-9525-65b04ea163ec None admin] PATH_INFO: `/v3/users/d65de4c60cb94982aef63c4d50fb8342` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:40.721679 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dbcea1-9ca3-43d3-9525-65b04ea163ec None admin] RBAC: Authorizing `identity:get_user(user_id=d65de4c60cb94982aef63c4d50fb8342)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:40.723211 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dbcea1-9ca3-43d3-9525-65b04ea163ec None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:40.724990 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 146/286] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 15 19:14:40 2022] GET /identity/v3/users/d65de4c60cb94982aef63c4d50fb8342 => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:40.725809 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dbcea1-9ca3-43d3-9525-65b04ea163ec None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:40.747115 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:40.814664 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427e013a-f951-4467-b992-c044d69d16ec None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:40.847585 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:40.858720 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dbcea1-9ca3-43d3-9525-65b04ea163ec None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:40.860866 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385eed85-85eb-4f53-859b-36ebdd1b31dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:40.861859 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385eed85-85eb-4f53-859b-36ebdd1b31dd None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:40.862043 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385eed85-85eb-4f53-859b-36ebdd1b31dd None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:40.862400 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385eed85-85eb-4f53-859b-36ebdd1b31dd None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:40.863225 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385eed85-85eb-4f53-859b-36ebdd1b31dd None admin] RBAC: Adding query filter params (name=member) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:40.863531 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385eed85-85eb-4f53-859b-36ebdd1b31dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:40.864909 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385eed85-85eb-4f53-859b-36ebdd1b31dd None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:40.877825 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 141/287] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jul 15 19:14:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:40.877825 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385eed85-85eb-4f53-859b-36ebdd1b31dd None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:40.905798 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b66fc07-11c1-43e3-b76e-d74bf34fcc68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:40.906663 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b66fc07-11c1-43e3-b76e-d74bf34fcc68 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:40.906942 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b66fc07-11c1-43e3-b76e-d74bf34fcc68 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:40.907216 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b66fc07-11c1-43e3-b76e-d74bf34fcc68 None admin] PATH_INFO: `/v3/projects/12e6f4bc187942989f83aeb1902a3036` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:40.908405 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b66fc07-11c1-43e3-b76e-d74bf34fcc68 None admin] RBAC: Authorizing `identity:get_project(project_id=12e6f4bc187942989f83aeb1902a3036)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:40.910938 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b66fc07-11c1-43e3-b76e-d74bf34fcc68 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:40.911593 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 147/288] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 15 19:14:40 2022] GET /identity/v3/projects/12e6f4bc187942989f83aeb1902a3036 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:40.912448 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b66fc07-11c1-43e3-b76e-d74bf34fcc68 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:41.024920 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385eed85-85eb-4f53-859b-36ebdd1b31dd None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:41.027816 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b66fc07-11c1-43e3-b76e-d74bf34fcc68 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:41.036645 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:41.064479 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0dba55-d855-4f99-9958-6ac5f3e129a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:41.065497 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0dba55-d855-4f99-9958-6ac5f3e129a3 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:41.065906 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0dba55-d855-4f99-9958-6ac5f3e129a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:41.066252 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0dba55-d855-4f99-9958-6ac5f3e129a3 None admin] PATH_INFO: `/v3/users/605a9210f12a47ec8f5769bd86b7c636` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:41.067535 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0dba55-d855-4f99-9958-6ac5f3e129a3 None admin] RBAC: Authorizing `identity:get_user(user_id=605a9210f12a47ec8f5769bd86b7c636)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:41.069048 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0dba55-d855-4f99-9958-6ac5f3e129a3 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:41.069297 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f649d3b6-57f9-4a70-aa0d-174e932d8540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:41.070076 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f649d3b6-57f9-4a70-aa0d-174e932d8540 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:41.070456 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f649d3b6-57f9-4a70-aa0d-174e932d8540 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:41.070796 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f649d3b6-57f9-4a70-aa0d-174e932d8540 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:41.071177 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 148/289] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 15 19:14:40 2022] GET /identity/v3/users/605a9210f12a47ec8f5769bd86b7c636 => generated 297 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:41.071768 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f649d3b6-57f9-4a70-aa0d-174e932d8540 None admin] RBAC: Adding query filter params (role.id=28763c6e70094116b3a4b2895895fa58, user.id=d65de4c60cb94982aef63c4d50fb8342, scope.system=all) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:41.072113 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0dba55-d855-4f99-9958-6ac5f3e129a3 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:41.076846 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f649d3b6-57f9-4a70-aa0d-174e932d8540 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:41.080536 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f649d3b6-57f9-4a70-aa0d-174e932d8540 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:41.089249 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 142/290] 10.222.0.33 () {56 vars in 1308 bytes} [Fri Jul 15 19:14:40 2022] GET /identity/v3/role_assignments?user.id=d65de4c60cb94982aef63c4d50fb8342&scope.system=all&role.id=28763c6e70094116b3a4b2895895fa58 => generated 226 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:41.090981 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f649d3b6-57f9-4a70-aa0d-174e932d8540 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:41.134088 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:41.274655 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f649d3b6-57f9-4a70-aa0d-174e932d8540 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:41.290519 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0dba55-d855-4f99-9958-6ac5f3e129a3 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:41.323135 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973d2d4c-8bee-4858-9839-0ad437d52926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:41.323799 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973d2d4c-8bee-4858-9839-0ad437d52926 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:41.324081 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973d2d4c-8bee-4858-9839-0ad437d52926 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:41.324361 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973d2d4c-8bee-4858-9839-0ad437d52926 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:41.338922 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2d1f67-d6ce-43cb-b8be-e4c1f7b7eea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:41.339821 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2d1f67-d6ce-43cb-b8be-e4c1f7b7eea5 None admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:41.340104 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2d1f67-d6ce-43cb-b8be-e4c1f7b7eea5 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:41.340381 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2d1f67-d6ce-43cb-b8be-e4c1f7b7eea5 None admin] PATH_INFO: `/v3/projects/12e6f4bc187942989f83aeb1902a3036/users/3cbee5c59e184e51baa688dc2aeadf36/roles/2f62da1ebb2b443eabf21e910a215048` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:41.344123 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2d1f67-d6ce-43cb-b8be-e4c1f7b7eea5 None admin] RBAC: Authorizing `identity:create_grant(project_id=12e6f4bc187942989f83aeb1902a3036, user_id=3cbee5c59e184e51baa688dc2aeadf36, role_id=2f62da1ebb2b443eabf21e910a215048)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:41.345574 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2d1f67-d6ce-43cb-b8be-e4c1f7b7eea5 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:41.381643 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:41.410659 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973d2d4c-8bee-4858-9839-0ad437d52926 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:41.412127 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973d2d4c-8bee-4858-9839-0ad437d52926 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:41.451303 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 149/291] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jul 15 19:14:41 2022] PUT /identity/v3/projects/12e6f4bc187942989f83aeb1902a3036/users/3cbee5c59e184e51baa688dc2aeadf36/roles/2f62da1ebb2b443eabf21e910a215048 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:14:41.534050 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 143/292] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 15 19:14:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 19:14:41.539542 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2d1f67-d6ce-43cb-b8be-e4c1f7b7eea5 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:41.659341 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:41.673863 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2d1f67-d6ce-43cb-b8be-e4c1f7b7eea5 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:41.714339 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a853351-03a1-422e-bbe8-9d14c94ae9e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:41.716062 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a853351-03a1-422e-bbe8-9d14c94ae9e8 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:41.716345 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a853351-03a1-422e-bbe8-9d14c94ae9e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:41.716621 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a853351-03a1-422e-bbe8-9d14c94ae9e8 None admin] PATH_INFO: `/v3/projects/12e6f4bc187942989f83aeb1902a3036` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:41.717954 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a853351-03a1-422e-bbe8-9d14c94ae9e8 None admin] RBAC: Authorizing `identity:get_project(project_id=12e6f4bc187942989f83aeb1902a3036)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:41.719429 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a853351-03a1-422e-bbe8-9d14c94ae9e8 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:41.721160 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 150/293] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 15 19:14:41 2022] GET /identity/v3/projects/12e6f4bc187942989f83aeb1902a3036 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:14:41.727827 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973d2d4c-8bee-4858-9839-0ad437d52926 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:41.802982 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:41.833053 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973d2d4c-8bee-4858-9839-0ad437d52926 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:41.879060 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635a4eca-372e-4c59-95e9-259ebf0cf380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:41.879727 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635a4eca-372e-4c59-95e9-259ebf0cf380 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:41.880011 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635a4eca-372e-4c59-95e9-259ebf0cf380 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:41.880283 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635a4eca-372e-4c59-95e9-259ebf0cf380 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:41.881122 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635a4eca-372e-4c59-95e9-259ebf0cf380 None admin] RBAC: Adding query filter params (role.id=28763c6e70094116b3a4b2895895fa58, scope.project.id=12e6f4bc187942989f83aeb1902a3036, user.id=605a9210f12a47ec8f5769bd86b7c636) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:41.881431 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635a4eca-372e-4c59-95e9-259ebf0cf380 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:41.883008 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635a4eca-372e-4c59-95e9-259ebf0cf380 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:41.890882 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 144/294] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 15 19:14:41 2022] GET /identity/v3/role_assignments?user.id=605a9210f12a47ec8f5769bd86b7c636&scope.project.id=12e6f4bc187942989f83aeb1902a3036&role.id=28763c6e70094116b3a4b2895895fa58 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:42.877561 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ff71b2-a70b-46c8-9a5f-344920b33a22 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:42.877561 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ff71b2-a70b-46c8-9a5f-344920b33a22 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:42.877561 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ff71b2-a70b-46c8-9a5f-344920b33a22 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:42.877561 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 151/295] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:14:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:14:42.884162 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4392e4-1cc2-44b9-9baf-a5e001549b7a None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:42.885068 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4392e4-1cc2-44b9-9baf-a5e001549b7a None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:42.885558 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4392e4-1cc2-44b9-9baf-a5e001549b7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:42.897455 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cfbaae-625f-4feb-a639-89e7347d52e2 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:42.898111 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cfbaae-625f-4feb-a639-89e7347d52e2 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:42.899201 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cfbaae-625f-4feb-a639-89e7347d52e2 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:42.901057 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 152/296] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:14:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 19:14:42.903882 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-dd4392e4-1cc2-44b9-9baf-a5e001549b7a None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:14:42.905113 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-dd4392e4-1cc2-44b9-9baf-a5e001549b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:14:42.906560 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c787b1c8-fb47-45c2-8eec-6dd154f7f1b0 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:42.906852 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c787b1c8-fb47-45c2-8eec-6dd154f7f1b0 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:42.907133 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c787b1c8-fb47-45c2-8eec-6dd154f7f1b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:42.926262 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-c787b1c8-fb47-45c2-8eec-6dd154f7f1b0 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:14:42.927461 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-c787b1c8-fb47-45c2-8eec-6dd154f7f1b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:14:42.992023 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 145/297] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:14:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:14:43.006763 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 153/298] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:14:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:14:43.014689 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4392e4-1cc2-44b9-9baf-a5e001549b7a None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:43.034848 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c787b1c8-fb47-45c2-8eec-6dd154f7f1b0 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:43.091311 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:43.128760 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4392e4-1cc2-44b9-9baf-a5e001549b7a None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:43.155347 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:43.168853 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389fb217-4bd3-4b18-a57f-8d66314dc360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:43.170039 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389fb217-4bd3-4b18-a57f-8d66314dc360 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:43.170555 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389fb217-4bd3-4b18-a57f-8d66314dc360 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:43.171032 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389fb217-4bd3-4b18-a57f-8d66314dc360 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:43.172316 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c787b1c8-fb47-45c2-8eec-6dd154f7f1b0 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:43.172474 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389fb217-4bd3-4b18-a57f-8d66314dc360 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:43.174026 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389fb217-4bd3-4b18-a57f-8d66314dc360 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:43.180358 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 146/299] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:14:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:43.182680 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389fb217-4bd3-4b18-a57f-8d66314dc360 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:43.217458 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d92808-1972-4931-95d4-66b1fce5ce08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:43.218658 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d92808-1972-4931-95d4-66b1fce5ce08 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:43.219137 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d92808-1972-4931-95d4-66b1fce5ce08 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:43.219641 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d92808-1972-4931-95d4-66b1fce5ce08 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:43.230107 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d92808-1972-4931-95d4-66b1fce5ce08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:14:43.238080 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-64d92808-1972-4931-95d4-66b1fce5ce08 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 19:14:43.238080 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:14:43.238080 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:14:43.238080 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:14:43.238080 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:14:43.238080 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:14:43.238080 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:14:43.238080 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:14:43.238080 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:14:43.238080 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:14:43.238080 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:14:43.238080 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:14:43.238080 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:14:43.238080 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:14:43.238080 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:14:43.238080 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:14:43.238937 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:14:43.238937 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:14:43.238937 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:14:43.238937 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:14:43.238937 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:14:43.238937 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:14:43.238937 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:14:43.238937 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:14:43.238937 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:14:43.238937 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:14:43.238937 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:14:43.238937 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:14:43.238937 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:14:43.238937 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:14:43.238937 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:14:43.238937 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:14:43.239731 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:14:43.239731 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:14:43.239731 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:14:43.239731 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:14:43.239731 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:14:43.239731 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:14:43.239731 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:14:43.239731 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:14:43.239731 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:14:43.239731 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:14:43.239731 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 19:14:43.239731 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:14:43.243136 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 154/300] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 15 19:14:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:14:43.249464 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d92808-1972-4931-95d4-66b1fce5ce08 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:43.309279 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389fb217-4bd3-4b18-a57f-8d66314dc360 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:43.346015 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ad6165-46d5-43dc-b045-06fe51656814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:43.347039 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ad6165-46d5-43dc-b045-06fe51656814 None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:43.347466 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ad6165-46d5-43dc-b045-06fe51656814 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:43.347881 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ad6165-46d5-43dc-b045-06fe51656814 None admin] PATH_INFO: `/v3/groups` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:43.348715 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ad6165-46d5-43dc-b045-06fe51656814 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:43.350361 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ad6165-46d5-43dc-b045-06fe51656814 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:43.354160 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:43.382994 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d92808-1972-4931-95d4-66b1fce5ce08 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:43.405715 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 147/301] 10.222.0.33 () {60 vars in 1134 bytes} [Fri Jul 15 19:14:43 2022] POST /identity/v3/groups => generated 229 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:14:43.407445 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52b0030-f92e-4b77-b553-b911718e0e32 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:43.407742 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52b0030-f92e-4b77-b553-b911718e0e32 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:43.408015 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52b0030-f92e-4b77-b553-b911718e0e32 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:43.409317 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 148/302] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:14:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:14:43.416233 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053c57c4-6d90-43f3-8c79-2bf7f0fc28bc None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:43.416580 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053c57c4-6d90-43f3-8c79-2bf7f0fc28bc None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:43.416944 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053c57c4-6d90-43f3-8c79-2bf7f0fc28bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:43.420961 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b25b5c-9bf1-463b-97fe-b51cbc8b1948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:43.422019 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b25b5c-9bf1-463b-97fe-b51cbc8b1948 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:43.422495 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b25b5c-9bf1-463b-97fe-b51cbc8b1948 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:43.422956 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b25b5c-9bf1-463b-97fe-b51cbc8b1948 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:43.423982 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b25b5c-9bf1-463b-97fe-b51cbc8b1948 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:43.424509 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b25b5c-9bf1-463b-97fe-b51cbc8b1948 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:43.426182 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b25b5c-9bf1-463b-97fe-b51cbc8b1948 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:43.440970 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-053c57c4-6d90-43f3-8c79-2bf7f0fc28bc None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:14:43.440970 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-053c57c4-6d90-43f3-8c79-2bf7f0fc28bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:14:43.441651 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 155/303] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jul 15 19:14:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:43.442857 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b25b5c-9bf1-463b-97fe-b51cbc8b1948 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:43.455707 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:43.520541 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 149/304] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:14:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 19:14:43.545421 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053c57c4-6d90-43f3-8c79-2bf7f0fc28bc None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:43.559719 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b25b5c-9bf1-463b-97fe-b51cbc8b1948 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:43.600564 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac12bcdc-8d09-45d4-b348-d6d7cad9ef50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:43.602714 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac12bcdc-8d09-45d4-b348-d6d7cad9ef50 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:43.603287 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac12bcdc-8d09-45d4-b348-d6d7cad9ef50 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:43.603575 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac12bcdc-8d09-45d4-b348-d6d7cad9ef50 None admin] PATH_INFO: `/v3/users/6ecf9665e7f1468ba57248f067821b3e` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:43.619984 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac12bcdc-8d09-45d4-b348-d6d7cad9ef50 None admin] RBAC: Authorizing `identity:get_user(user_id=6ecf9665e7f1468ba57248f067821b3e)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:43.621565 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac12bcdc-8d09-45d4-b348-d6d7cad9ef50 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:43.623519 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 156/305] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 15 19:14:43 2022] GET /identity/v3/users/6ecf9665e7f1468ba57248f067821b3e => generated 293 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:43.628910 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac12bcdc-8d09-45d4-b348-d6d7cad9ef50 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:43.671228 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053c57c4-6d90-43f3-8c79-2bf7f0fc28bc None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:43.708147 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:43.708821 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:43.724051 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d93f53-2374-47d8-a4d4-8447888451c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:43.725472 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d93f53-2374-47d8-a4d4-8447888451c8 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:43.725988 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d93f53-2374-47d8-a4d4-8447888451c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:43.726600 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d93f53-2374-47d8-a4d4-8447888451c8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:43.742052 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d93f53-2374-47d8-a4d4-8447888451c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:14:43.747604 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac12bcdc-8d09-45d4-b348-d6d7cad9ef50 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:43.755020 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-03d93f53-2374-47d8-a4d4-8447888451c8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 19:14:43.755020 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:14:43.755020 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:14:43.755020 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:14:43.755020 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:14:43.755020 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:14:43.755020 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:14:43.755020 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:14:43.755020 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:14:43.755020 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:14:43.755020 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:14:43.755020 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:14:43.755020 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:14:43.755020 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:14:43.755020 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:14:43.755020 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:14:43.756214 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:14:43.756214 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:14:43.756214 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:14:43.756214 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:14:43.756214 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:14:43.756214 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:14:43.756214 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:14:43.756214 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:14:43.756214 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:14:43.756214 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:14:43.756214 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:14:43.756214 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:14:43.756214 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:14:43.756214 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:14:43.756214 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:14:43.756214 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:14:43.758394 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:14:43.758394 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:14:43.758394 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:14:43.758394 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:14:43.758394 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:14:43.758394 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:14:43.758394 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:14:43.758394 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:14:43.758394 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:14:43.758394 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:14:43.758394 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 19:14:43.758394 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:14:43.760500 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 150/306] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 15 19:14:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:14:43.767537 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d93f53-2374-47d8-a4d4-8447888451c8 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:43.808745 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cfbc63-5f86-406f-a5ed-f8e3490271cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:43.809440 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cfbc63-5f86-406f-a5ed-f8e3490271cb None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:43.809727 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cfbc63-5f86-406f-a5ed-f8e3490271cb None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:43.811529 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cfbc63-5f86-406f-a5ed-f8e3490271cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:43.812381 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cfbc63-5f86-406f-a5ed-f8e3490271cb None admin] RBAC: Adding query filter params (role.id=2f62da1ebb2b443eabf21e910a215048, user.id=6ecf9665e7f1468ba57248f067821b3e, scope.system=all) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:43.812699 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cfbc63-5f86-406f-a5ed-f8e3490271cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:43.814227 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cfbc63-5f86-406f-a5ed-f8e3490271cb None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:43.819797 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 157/307] 10.222.0.33 () {56 vars in 1308 bytes} [Fri Jul 15 19:14:43 2022] GET /identity/v3/role_assignments?user.id=6ecf9665e7f1468ba57248f067821b3e&scope.system=all&role.id=2f62da1ebb2b443eabf21e910a215048 => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:14:43.896328 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d93f53-2374-47d8-a4d4-8447888451c8 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:43.942357 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa05b06-8cbd-4aaa-82c6-e8c157db9f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:43.943043 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa05b06-8cbd-4aaa-82c6-e8c157db9f83 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:43.943329 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa05b06-8cbd-4aaa-82c6-e8c157db9f83 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:43.943610 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa05b06-8cbd-4aaa-82c6-e8c157db9f83 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:43.944379 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa05b06-8cbd-4aaa-82c6-e8c157db9f83 None admin] RBAC: Adding query filter params (name=member) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:43.944765 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa05b06-8cbd-4aaa-82c6-e8c157db9f83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:43.946999 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa05b06-8cbd-4aaa-82c6-e8c157db9f83 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:43.956899 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 151/308] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jul 15 19:14:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:14:43.963682 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cfbc63-5f86-406f-a5ed-f8e3490271cb None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:44.018055 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0a8567-46ba-414d-97e3-852584ab66af None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:44.018387 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0a8567-46ba-414d-97e3-852584ab66af None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:44.018624 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0a8567-46ba-414d-97e3-852584ab66af None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:44.020189 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 152/309] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:14:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:14:44.026450 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6b83fc-01a3-45b9-9db5-78d5e0575ac0 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:44.027017 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6b83fc-01a3-45b9-9db5-78d5e0575ac0 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:44.027489 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6b83fc-01a3-45b9-9db5-78d5e0575ac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:44.045511 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-ea6b83fc-01a3-45b9-9db5-78d5e0575ac0 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:14:44.047098 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-ea6b83fc-01a3-45b9-9db5-78d5e0575ac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:14:44.074644 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cfbc63-5f86-406f-a5ed-f8e3490271cb None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:44.097034 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:44.104899 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:44.110713 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21909918-47ab-4e22-845d-15e21929be9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:44.111748 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21909918-47ab-4e22-845d-15e21929be9f None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:44.112517 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21909918-47ab-4e22-845d-15e21929be9f None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:44.113100 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21909918-47ab-4e22-845d-15e21929be9f None admin] PATH_INFO: `/v3/users/d65de4c60cb94982aef63c4d50fb8342` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:44.114762 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21909918-47ab-4e22-845d-15e21929be9f None admin] RBAC: Authorizing `identity:get_user(user_id=d65de4c60cb94982aef63c4d50fb8342)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:44.117554 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21909918-47ab-4e22-845d-15e21929be9f None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:44.120469 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 158/310] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 15 19:14:43 2022] GET /identity/v3/users/d65de4c60cb94982aef63c4d50fb8342 => generated 293 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:44.126052 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21909918-47ab-4e22-845d-15e21929be9f None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:44.133952 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 153/311] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:14:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 19:14:44.152852 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6b83fc-01a3-45b9-9db5-78d5e0575ac0 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:44.252389 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21909918-47ab-4e22-845d-15e21929be9f None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:44.283387 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6b83fc-01a3-45b9-9db5-78d5e0575ac0 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:44.289178 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1d4892-79d3-462d-8804-7f13b899cd89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:44.290333 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d4892-79d3-462d-8804-7f13b899cd89 None admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:44.290837 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d4892-79d3-462d-8804-7f13b899cd89 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:44.291384 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d4892-79d3-462d-8804-7f13b899cd89 None admin] PATH_INFO: `/v3/system/users/d65de4c60cb94982aef63c4d50fb8342/roles/28763c6e70094116b3a4b2895895fa58` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:44.293147 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1d4892-79d3-462d-8804-7f13b899cd89 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=d65de4c60cb94982aef63c4d50fb8342, role_id=28763c6e70094116b3a4b2895895fa58)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:44.294747 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1d4892-79d3-462d-8804-7f13b899cd89 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:44.322080 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c11cb38-6a2f-4c8e-adbb-783f73fc31b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:44.323043 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c11cb38-6a2f-4c8e-adbb-783f73fc31b2 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:44.323329 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c11cb38-6a2f-4c8e-adbb-783f73fc31b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:44.323653 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c11cb38-6a2f-4c8e-adbb-783f73fc31b2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:44.335357 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c11cb38-6a2f-4c8e-adbb-783f73fc31b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:14:44.343201 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-9c11cb38-6a2f-4c8e-adbb-783f73fc31b2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 19:14:44.343201 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:14:44.343201 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:14:44.343201 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:14:44.343201 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:14:44.343201 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:14:44.343201 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:14:44.343201 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:14:44.343201 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:14:44.343201 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:14:44.343201 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:14:44.343201 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:14:44.343201 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:14:44.343201 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:14:44.343201 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:14:44.343201 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:14:44.344134 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:14:44.344134 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:14:44.344134 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:14:44.344134 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:14:44.344134 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:14:44.344134 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:14:44.344134 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:14:44.344134 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:14:44.344134 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:14:44.344134 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:14:44.344134 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:14:44.344134 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:14:44.344134 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:14:44.344134 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:14:44.344134 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:14:44.344134 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:14:44.344993 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:14:44.344993 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:14:44.344993 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:14:44.344993 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:14:44.344993 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:14:44.344993 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:14:44.344993 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:14:44.344993 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:14:44.344993 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:14:44.344993 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:14:44.344993 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 19:14:44.344993 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:14:44.344993 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 154/312] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 15 19:14:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:14:44.352209 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c11cb38-6a2f-4c8e-adbb-783f73fc31b2 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:44.390908 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:44.405871 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 159/313] 10.222.0.33 () {58 vars in 1334 bytes} [Fri Jul 15 19:14:44 2022] PUT /identity/v3/system/users/d65de4c60cb94982aef63c4d50fb8342/roles/28763c6e70094116b3a4b2895895fa58 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:14:44.473316 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c11cb38-6a2f-4c8e-adbb-783f73fc31b2 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:44.513488 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cccfd8c-c378-425b-9dce-e1d2e8a01fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:44.514267 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cccfd8c-c378-425b-9dce-e1d2e8a01fa2 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:44.514555 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cccfd8c-c378-425b-9dce-e1d2e8a01fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:44.514836 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cccfd8c-c378-425b-9dce-e1d2e8a01fa2 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:44.515664 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cccfd8c-c378-425b-9dce-e1d2e8a01fa2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:44.515975 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cccfd8c-c378-425b-9dce-e1d2e8a01fa2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:44.517467 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cccfd8c-c378-425b-9dce-e1d2e8a01fa2 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:44.528263 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 155/314] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jul 15 19:14:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:14:44.537800 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1d4892-79d3-462d-8804-7f13b899cd89 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:44.607906 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:44.652165 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1d4892-79d3-462d-8804-7f13b899cd89 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:44.687717 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7716ec46-c0af-49df-bd2f-8c1699a29b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:44.688936 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7716ec46-c0af-49df-bd2f-8c1699a29b4a None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:44.689414 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7716ec46-c0af-49df-bd2f-8c1699a29b4a None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:44.689925 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7716ec46-c0af-49df-bd2f-8c1699a29b4a None admin] PATH_INFO: `/v3/users/3cbee5c59e184e51baa688dc2aeadf36` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:44.691323 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7716ec46-c0af-49df-bd2f-8c1699a29b4a None admin] RBAC: Authorizing `identity:get_user(user_id=3cbee5c59e184e51baa688dc2aeadf36)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:44.692959 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7716ec46-c0af-49df-bd2f-8c1699a29b4a None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:44.695738 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 160/315] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 15 19:14:44 2022] GET /identity/v3/users/3cbee5c59e184e51baa688dc2aeadf36 => generated 297 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:44.705789 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cccfd8c-c378-425b-9dce-e1d2e8a01fa2 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:44.803854 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:44.810710 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cccfd8c-c378-425b-9dce-e1d2e8a01fa2 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:44.846206 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65afca49-8d9d-4bee-bab2-391a7391d2cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:44.846863 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65afca49-8d9d-4bee-bab2-391a7391d2cd None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:44.847148 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65afca49-8d9d-4bee-bab2-391a7391d2cd None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:44.847424 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65afca49-8d9d-4bee-bab2-391a7391d2cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:44.918070 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65afca49-8d9d-4bee-bab2-391a7391d2cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:44.919540 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65afca49-8d9d-4bee-bab2-391a7391d2cd None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:45.028942 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 156/316] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 15 19:14:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 19:14:45.029408 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:45.041803 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7716ec46-c0af-49df-bd2f-8c1699a29b4a None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:45.139810 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7716ec46-c0af-49df-bd2f-8c1699a29b4a None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:45.169639 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:45.175567 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef41de4-de76-44b8-b083-5929954c6080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:45.176442 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef41de4-de76-44b8-b083-5929954c6080 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:45.183371 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef41de4-de76-44b8-b083-5929954c6080 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:45.183371 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef41de4-de76-44b8-b083-5929954c6080 None admin] PATH_INFO: `/v3/projects/12e6f4bc187942989f83aeb1902a3036` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:45.183371 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef41de4-de76-44b8-b083-5929954c6080 None admin] RBAC: Authorizing `identity:get_project(project_id=12e6f4bc187942989f83aeb1902a3036)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:45.183371 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef41de4-de76-44b8-b083-5929954c6080 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:45.187760 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 161/317] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 15 19:14:45 2022] GET /identity/v3/projects/12e6f4bc187942989f83aeb1902a3036 => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:45.189119 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65afca49-8d9d-4bee-bab2-391a7391d2cd None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:45.301320 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65afca49-8d9d-4bee-bab2-391a7391d2cd None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:45.338739 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7873c0-bff2-48a4-bf7e-f9ca86cd7c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:45.339793 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7873c0-bff2-48a4-bf7e-f9ca86cd7c15 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:45.340257 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7873c0-bff2-48a4-bf7e-f9ca86cd7c15 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:45.340711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7873c0-bff2-48a4-bf7e-f9ca86cd7c15 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:45.341725 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7873c0-bff2-48a4-bf7e-f9ca86cd7c15 None admin] RBAC: Adding query filter params (role.id=2f62da1ebb2b443eabf21e910a215048, scope.project.id=12e6f4bc187942989f83aeb1902a3036, user.id=3cbee5c59e184e51baa688dc2aeadf36) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:45.342264 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7873c0-bff2-48a4-bf7e-f9ca86cd7c15 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:45.343888 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7873c0-bff2-48a4-bf7e-f9ca86cd7c15 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:45.350547 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 157/318] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 15 19:14:45 2022] GET /identity/v3/role_assignments?user.id=3cbee5c59e184e51baa688dc2aeadf36&scope.project.id=12e6f4bc187942989f83aeb1902a3036&role.id=2f62da1ebb2b443eabf21e910a215048 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:45.630436 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4556f705-7fd2-40b0-9be0-8aa3e719aedf None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:45.630809 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4556f705-7fd2-40b0-9be0-8aa3e719aedf None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:45.631008 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4556f705-7fd2-40b0-9be0-8aa3e719aedf None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:45.633093 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 162/319] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:14:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:14:45.638159 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd8b81-9ca5-41f3-9808-7c2e9ba219a1 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:45.638456 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd8b81-9ca5-41f3-9808-7c2e9ba219a1 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:45.638732 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd8b81-9ca5-41f3-9808-7c2e9ba219a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:45.655990 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-c0dd8b81-9ca5-41f3-9808-7c2e9ba219a1 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:14:45.657186 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-c0dd8b81-9ca5-41f3-9808-7c2e9ba219a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:14:45.716430 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:45.727130 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 158/320] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:14:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:14:45.748896 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4556f705-7fd2-40b0-9be0-8aa3e719aedf None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:45.860563 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4556f705-7fd2-40b0-9be0-8aa3e719aedf None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:45.895143 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b33aaf-d640-4704-910e-f9d0510ab3b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:45.895865 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b33aaf-d640-4704-910e-f9d0510ab3b7 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:45.896148 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b33aaf-d640-4704-910e-f9d0510ab3b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:45.896425 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b33aaf-d640-4704-910e-f9d0510ab3b7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:45.897600 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b33aaf-d640-4704-910e-f9d0510ab3b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:45.899371 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b33aaf-d640-4704-910e-f9d0510ab3b7 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:45.901151 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 163/321] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:14:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:45.906571 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dd8b81-9ca5-41f3-9808-7c2e9ba219a1 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:45.939191 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6dfe47-ca2c-4e03-afc7-eb3012ff3592 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:45.939516 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6dfe47-ca2c-4e03-afc7-eb3012ff3592 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:45.939748 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6dfe47-ca2c-4e03-afc7-eb3012ff3592 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:45.941254 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 164/322] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:14:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 19:14:45.946959 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd74eec0-d982-4f3e-b7cc-fc67e63ce1cb None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:45.947249 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd74eec0-d982-4f3e-b7cc-fc67e63ce1cb None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:45.947530 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd74eec0-d982-4f3e-b7cc-fc67e63ce1cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:45.964452 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-fd74eec0-d982-4f3e-b7cc-fc67e63ce1cb None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:14:45.965641 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-fd74eec0-d982-4f3e-b7cc-fc67e63ce1cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:14:46.010242 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dd8b81-9ca5-41f3-9808-7c2e9ba219a1 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:46.030924 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:46.038169 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 165/323] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:14:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 19:14:46.052882 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4176f321-9f19-4939-b073-7ab61e35ee9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:46.053609 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4176f321-9f19-4939-b073-7ab61e35ee9d None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:46.053929 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4176f321-9f19-4939-b073-7ab61e35ee9d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:46.054777 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4176f321-9f19-4939-b073-7ab61e35ee9d None admin] PATH_INFO: `/v3/groups` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:46.055488 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4176f321-9f19-4939-b073-7ab61e35ee9d None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:46.056990 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4176f321-9f19-4939-b073-7ab61e35ee9d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:46.062366 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd74eec0-d982-4f3e-b7cc-fc67e63ce1cb None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:46.160289 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd74eec0-d982-4f3e-b7cc-fc67e63ce1cb None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:46.188109 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 159/324] 10.222.0.33 () {60 vars in 1134 bytes} [Fri Jul 15 19:14:45 2022] POST /identity/v3/groups => generated 226 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:14:46.208499 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1e9346-f572-4ca7-bba0-0d45a92a45ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:46.209343 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1e9346-f572-4ca7-bba0-0d45a92a45ed None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:46.209619 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1e9346-f572-4ca7-bba0-0d45a92a45ed None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:46.209929 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1e9346-f572-4ca7-bba0-0d45a92a45ed None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:46.222913 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1e9346-f572-4ca7-bba0-0d45a92a45ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:14:46.231468 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-fb1e9346-f572-4ca7-bba0-0d45a92a45ed None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 19:14:46.231468 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:14:46.231468 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:14:46.231468 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:14:46.231468 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:14:46.231468 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:14:46.231468 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:14:46.231468 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:14:46.231468 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:14:46.231468 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:14:46.231468 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:14:46.231468 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:14:46.231468 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:14:46.231468 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:14:46.231468 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:14:46.231468 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:14:46.232397 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:14:46.232397 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:14:46.232397 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:14:46.232397 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:14:46.232397 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:14:46.232397 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:14:46.232397 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:14:46.232397 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:14:46.232397 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:14:46.232397 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:14:46.232397 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:14:46.232397 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:14:46.232397 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:14:46.232397 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:14:46.232397 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:14:46.232397 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:14:46.233201 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:14:46.233201 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:14:46.233201 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:14:46.233201 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:14:46.233201 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:14:46.233201 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:14:46.233201 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:14:46.233201 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:14:46.233201 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:14:46.233201 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:14:46.233201 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 19:14:46.233201 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:14:46.233201 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 166/325] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 15 19:14:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:14:46.239056 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4176f321-9f19-4939-b073-7ab61e35ee9d None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:46.305513 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:46.358142 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4176f321-9f19-4939-b073-7ab61e35ee9d None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:46.432824 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb729c45-386b-42b5-9875-b99a7c781f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:46.433892 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb729c45-386b-42b5-9875-b99a7c781f35 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:46.434508 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb729c45-386b-42b5-9875-b99a7c781f35 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:46.434996 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb729c45-386b-42b5-9875-b99a7c781f35 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:46.435967 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb729c45-386b-42b5-9875-b99a7c781f35 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:46.436493 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb729c45-386b-42b5-9875-b99a7c781f35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:46.438195 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb729c45-386b-42b5-9875-b99a7c781f35 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:46.448889 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 160/326] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jul 15 19:14:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:46.454792 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1e9346-f572-4ca7-bba0-0d45a92a45ed None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:46.496008 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2f74e5-8240-4246-98b8-0c443c422ca8 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:46.496333 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2f74e5-8240-4246-98b8-0c443c422ca8 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:46.496562 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2f74e5-8240-4246-98b8-0c443c422ca8 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:46.498222 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 161/327] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:14:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:14:46.503395 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30904970-1344-485d-bedf-0b1394779b8a None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:46.503678 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30904970-1344-485d-bedf-0b1394779b8a None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:46.503952 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30904970-1344-485d-bedf-0b1394779b8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:46.516305 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:46.524211 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-30904970-1344-485d-bedf-0b1394779b8a None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:14:46.525701 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-30904970-1344-485d-bedf-0b1394779b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:14:46.573697 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1e9346-f572-4ca7-bba0-0d45a92a45ed None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:46.614860 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2238004-ee78-49d2-9900-345d0e30a33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:46.615840 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2238004-ee78-49d2-9900-345d0e30a33a None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:46.616196 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2238004-ee78-49d2-9900-345d0e30a33a None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:46.616527 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2238004-ee78-49d2-9900-345d0e30a33a None admin] PATH_INFO: `/v3/users/6ecf9665e7f1468ba57248f067821b3e` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:46.618430 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2238004-ee78-49d2-9900-345d0e30a33a None admin] RBAC: Authorizing `identity:get_user(user_id=6ecf9665e7f1468ba57248f067821b3e)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:46.619229 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 162/328] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:14:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:14:46.619889 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2238004-ee78-49d2-9900-345d0e30a33a None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:46.621650 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 167/329] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 15 19:14:46 2022] GET /identity/v3/users/6ecf9665e7f1468ba57248f067821b3e => generated 293 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:46.627709 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30904970-1344-485d-bedf-0b1394779b8a None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:46.647451 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2238004-ee78-49d2-9900-345d0e30a33a None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:46.684203 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:46.751329 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30904970-1344-485d-bedf-0b1394779b8a None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:46.790940 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd886a58-1166-44e0-871b-4a19c1f7445d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:46.791818 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd886a58-1166-44e0-871b-4a19c1f7445d None admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:46.792095 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd886a58-1166-44e0-871b-4a19c1f7445d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:46.792369 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd886a58-1166-44e0-871b-4a19c1f7445d None admin] PATH_INFO: `/v3/system/users/6ecf9665e7f1468ba57248f067821b3e/roles/2f62da1ebb2b443eabf21e910a215048` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:46.793493 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2238004-ee78-49d2-9900-345d0e30a33a None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:46.793913 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd886a58-1166-44e0-871b-4a19c1f7445d None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=6ecf9665e7f1468ba57248f067821b3e, role_id=2f62da1ebb2b443eabf21e910a215048)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:46.795306 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd886a58-1166-44e0-871b-4a19c1f7445d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:46.826162 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:46.839094 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5be93a3-9838-4c5f-a3f8-79353f837109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:46.839971 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5be93a3-9838-4c5f-a3f8-79353f837109 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:46.840258 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5be93a3-9838-4c5f-a3f8-79353f837109 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:46.840578 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5be93a3-9838-4c5f-a3f8-79353f837109 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:46.852941 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5be93a3-9838-4c5f-a3f8-79353f837109 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:14:46.860987 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-c5be93a3-9838-4c5f-a3f8-79353f837109 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 19:14:46.860987 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:14:46.860987 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:14:46.860987 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:14:46.860987 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:14:46.860987 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:14:46.860987 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:14:46.860987 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:14:46.860987 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:14:46.860987 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:14:46.860987 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:14:46.860987 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:14:46.860987 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:14:46.860987 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:14:46.860987 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:14:46.860987 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:14:46.861922 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:14:46.861922 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:14:46.861922 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:14:46.861922 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:14:46.861922 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:14:46.861922 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:14:46.861922 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:14:46.861922 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:14:46.861922 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:14:46.861922 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:14:46.861922 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:14:46.861922 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:14:46.861922 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:14:46.861922 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:14:46.861922 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:14:46.861922 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:14:46.862741 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:14:46.862741 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:14:46.862741 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:14:46.862741 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:14:46.862741 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:14:46.862741 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:14:46.862741 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:14:46.862741 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:14:46.862741 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:14:46.862741 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:14:46.862741 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 19:14:46.862741 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:14:46.862741 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 168/330] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 15 19:14:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:14:46.868604 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 163/331] 10.222.0.33 () {58 vars in 1334 bytes} [Fri Jul 15 19:14:46 2022] PUT /identity/v3/system/users/6ecf9665e7f1468ba57248f067821b3e/roles/2f62da1ebb2b443eabf21e910a215048 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:14:46.872229 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5be93a3-9838-4c5f-a3f8-79353f837109 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:46.997235 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5be93a3-9838-4c5f-a3f8-79353f837109 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:47.037936 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21860d0-9173-472c-a18e-51c3d2f7fc0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:47.039210 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21860d0-9173-472c-a18e-51c3d2f7fc0d None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:47.039498 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21860d0-9173-472c-a18e-51c3d2f7fc0d None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:47.039771 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21860d0-9173-472c-a18e-51c3d2f7fc0d None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:47.040541 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21860d0-9173-472c-a18e-51c3d2f7fc0d None admin] RBAC: Adding query filter params (name=member) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:47.040846 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21860d0-9173-472c-a18e-51c3d2f7fc0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:47.045386 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21860d0-9173-472c-a18e-51c3d2f7fc0d None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:47.059484 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 169/332] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jul 15 19:14:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:47.065664 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd886a58-1166-44e0-871b-4a19c1f7445d None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:47.195659 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd886a58-1166-44e0-871b-4a19c1f7445d None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:47.210627 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:47.244218 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05486ee-be60-4b77-9056-e8c1769f7d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:47.245112 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05486ee-be60-4b77-9056-e8c1769f7d1b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:47.245395 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05486ee-be60-4b77-9056-e8c1769f7d1b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:47.245673 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05486ee-be60-4b77-9056-e8c1769f7d1b None admin] PATH_INFO: `/v3/users/d65de4c60cb94982aef63c4d50fb8342` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:47.248719 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05486ee-be60-4b77-9056-e8c1769f7d1b None admin] RBAC: Authorizing `identity:get_user(user_id=d65de4c60cb94982aef63c4d50fb8342)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:47.250218 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05486ee-be60-4b77-9056-e8c1769f7d1b None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:47.252262 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 164/333] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 15 19:14:47 2022] GET /identity/v3/users/d65de4c60cb94982aef63c4d50fb8342 => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:47.259729 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21860d0-9173-472c-a18e-51c3d2f7fc0d None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:47.341583 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:47.417313 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21860d0-9173-472c-a18e-51c3d2f7fc0d None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:47.458238 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1396ba05-f9c8-441f-ae86-e5823e69e2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:47.458901 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1396ba05-f9c8-441f-ae86-e5823e69e2ef None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:47.459187 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1396ba05-f9c8-441f-ae86-e5823e69e2ef None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:47.459470 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1396ba05-f9c8-441f-ae86-e5823e69e2ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:47.460320 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1396ba05-f9c8-441f-ae86-e5823e69e2ef None admin] RBAC: Adding query filter params (role.id=28763c6e70094116b3a4b2895895fa58, user.id=d65de4c60cb94982aef63c4d50fb8342, scope.system=all) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:47.461161 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1396ba05-f9c8-441f-ae86-e5823e69e2ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:47.462638 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1396ba05-f9c8-441f-ae86-e5823e69e2ef None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:47.481030 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 170/334] 10.222.0.33 () {56 vars in 1308 bytes} [Fri Jul 15 19:14:47 2022] GET /identity/v3/role_assignments?user.id=d65de4c60cb94982aef63c4d50fb8342&scope.system=all&role.id=28763c6e70094116b3a4b2895895fa58 => generated 510 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:48.946673 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17469a5-bf79-4857-abac-905d1294cad6 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:48.947240 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17469a5-bf79-4857-abac-905d1294cad6 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:48.947240 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17469a5-bf79-4857-abac-905d1294cad6 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:48.948760 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 165/335] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:14:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:14:48.954284 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff560431-a384-4e9e-92b3-dd7cf3b72455 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:48.955157 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff560431-a384-4e9e-92b3-dd7cf3b72455 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:48.955651 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff560431-a384-4e9e-92b3-dd7cf3b72455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:48.987220 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-ff560431-a384-4e9e-92b3-dd7cf3b72455 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:14:48.988432 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-ff560431-a384-4e9e-92b3-dd7cf3b72455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:14:49.073224 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 171/336] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:14:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 19:14:49.113989 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17469a5-bf79-4857-abac-905d1294cad6 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:49.254978 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17469a5-bf79-4857-abac-905d1294cad6 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:49.298477 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b027e51f-029f-4c99-96e3-1d299501e465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:49.299294 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b027e51f-029f-4c99-96e3-1d299501e465 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:49.299578 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b027e51f-029f-4c99-96e3-1d299501e465 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:49.299855 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b027e51f-029f-4c99-96e3-1d299501e465 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:49.314666 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b027e51f-029f-4c99-96e3-1d299501e465 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:14:49.325156 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-b027e51f-029f-4c99-96e3-1d299501e465 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 19:14:49.325156 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:14:49.325156 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:14:49.325156 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:14:49.325156 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:14:49.325156 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:14:49.325156 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:14:49.325156 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:14:49.325156 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:14:49.325156 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:14:49.325156 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:14:49.325156 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:14:49.325156 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:14:49.325156 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:14:49.325156 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:14:49.325156 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:14:49.327099 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:14:49.327099 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:14:49.327099 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:14:49.327099 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:14:49.327099 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:14:49.327099 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:14:49.327099 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:14:49.327099 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:14:49.327099 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:14:49.327099 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:14:49.327099 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:14:49.327099 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:14:49.327099 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:14:49.327099 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:14:49.327099 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:14:49.327099 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:14:49.327941 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:14:49.327941 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:14:49.327941 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:14:49.327941 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:14:49.327941 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:14:49.327941 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:14:49.327941 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:14:49.327941 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:14:49.327941 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:14:49.327941 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:14:49.327941 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 19:14:49.327941 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:14:49.330982 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 166/337] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 15 19:14:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:14:49.337198 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff560431-a384-4e9e-92b3-dd7cf3b72455 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:49.399570 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:49.483666 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff560431-a384-4e9e-92b3-dd7cf3b72455 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:49.497330 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bd33d5-44b5-4bf5-b606-7bd222a8bba8 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:49.497330 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bd33d5-44b5-4bf5-b606-7bd222a8bba8 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:49.497330 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bd33d5-44b5-4bf5-b606-7bd222a8bba8 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:49.498224 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 167/338] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:14:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:14:49.502538 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca4352c-afa1-4772-a810-5a48bd1cb19d None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:49.503077 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca4352c-afa1-4772-a810-5a48bd1cb19d None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:49.503568 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca4352c-afa1-4772-a810-5a48bd1cb19d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:49.525925 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d526af-1db7-48e6-a8b2-aa7245552d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:49.526660 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d526af-1db7-48e6-a8b2-aa7245552d7a None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:49.526948 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d526af-1db7-48e6-a8b2-aa7245552d7a None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:49.527218 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d526af-1db7-48e6-a8b2-aa7245552d7a None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:49.527972 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d526af-1db7-48e6-a8b2-aa7245552d7a None admin] RBAC: Adding query filter params (name=member) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:49.528275 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d526af-1db7-48e6-a8b2-aa7245552d7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:49.533809 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d526af-1db7-48e6-a8b2-aa7245552d7a None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:49.543442 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 172/339] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jul 15 19:14:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:49.550850 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-aca4352c-afa1-4772-a810-5a48bd1cb19d None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:14:49.552319 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-aca4352c-afa1-4772-a810-5a48bd1cb19d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:14:49.566865 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d526af-1db7-48e6-a8b2-aa7245552d7a None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:49.603555 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:49.661799 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 168/340] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:14:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:14:49.689065 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca4352c-afa1-4772-a810-5a48bd1cb19d None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:49.713816 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:49.720272 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d526af-1db7-48e6-a8b2-aa7245552d7a None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:49.771175 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd17855-acf6-429f-b730-20f7329d5be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:49.771848 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd17855-acf6-429f-b730-20f7329d5be4 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:49.772159 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd17855-acf6-429f-b730-20f7329d5be4 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:49.772458 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd17855-acf6-429f-b730-20f7329d5be4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:49.875746 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca4352c-afa1-4772-a810-5a48bd1cb19d None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:49.921089 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd17855-acf6-429f-b730-20f7329d5be4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:49.922562 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd17855-acf6-429f-b730-20f7329d5be4 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:49.944686 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a5c79d-c5e6-46a8-be66-feafdce51e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:49.946031 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a5c79d-c5e6-46a8-be66-feafdce51e5c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:49.946309 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a5c79d-c5e6-46a8-be66-feafdce51e5c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:49.946580 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a5c79d-c5e6-46a8-be66-feafdce51e5c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:49.969505 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a5c79d-c5e6-46a8-be66-feafdce51e5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:14:49.977337 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-65a5c79d-c5e6-46a8-be66-feafdce51e5c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 19:14:49.977337 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:14:49.977337 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:14:49.977337 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:14:49.977337 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:14:49.977337 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:14:49.977337 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:14:49.977337 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:14:49.977337 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:14:49.977337 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:14:49.977337 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:14:49.977337 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:14:49.977337 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:14:49.977337 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:14:49.977337 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:14:49.977337 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:14:49.978297 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:14:49.978297 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:14:49.978297 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:14:49.978297 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:14:49.978297 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:14:49.978297 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:14:49.978297 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:14:49.978297 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:14:49.978297 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:14:49.978297 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:14:49.978297 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:14:49.978297 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:14:49.978297 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:14:49.978297 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:14:49.978297 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:14:49.978297 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:14:49.979101 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:14:49.979101 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:14:49.979101 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:14:49.979101 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:14:49.979101 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:14:49.979101 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:14:49.979101 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:14:49.979101 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:14:49.979101 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:14:49.979101 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:14:49.979101 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 19:14:49.979101 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:14:49.985252 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 169/341] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 15 19:14:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:14:49.995156 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac83eee2-87bc-4e5c-a96b-5fbf2afdbc4a None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:49.996419 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac83eee2-87bc-4e5c-a96b-5fbf2afdbc4a None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:49.996859 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac83eee2-87bc-4e5c-a96b-5fbf2afdbc4a None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:50.002281 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 170/342] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:14:49 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:14:50.006147 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac83eee2-87bc-4e5c-a96b-5fbf2afdbc4a None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:50.067669 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:50.092173 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:50.100718 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 173/343] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 15 19:14:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 19:14:50.104846 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f50493-3fde-41c0-b916-d693a5a85769 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:50.105486 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f50493-3fde-41c0-b916-d693a5a85769 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:50.105962 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f50493-3fde-41c0-b916-d693a5a85769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:50.127658 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-01f50493-3fde-41c0-b916-d693a5a85769 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:14:50.129681 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-01f50493-3fde-41c0-b916-d693a5a85769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:14:50.214226 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 174/344] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:14:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:14:50.215324 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f50493-3fde-41c0-b916-d693a5a85769 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:50.257987 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac83eee2-87bc-4e5c-a96b-5fbf2afdbc4a None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:50.294660 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5272de-a0ac-4473-8771-7918b3a53875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:50.295331 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5272de-a0ac-4473-8771-7918b3a53875 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:50.295633 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5272de-a0ac-4473-8771-7918b3a53875 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:50.296195 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5272de-a0ac-4473-8771-7918b3a53875 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:50.296643 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5272de-a0ac-4473-8771-7918b3a53875 None admin] RBAC: Adding query filter params (name=member) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:50.296947 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5272de-a0ac-4473-8771-7918b3a53875 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:50.298452 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5272de-a0ac-4473-8771-7918b3a53875 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:50.309141 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 171/345] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jul 15 19:14:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:50.313786 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5272de-a0ac-4473-8771-7918b3a53875 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:50.346198 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f50493-3fde-41c0-b916-d693a5a85769 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:50.378285 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:50.392572 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20030f32-8a42-42d0-8cdc-5afec18b9b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:50.393446 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20030f32-8a42-42d0-8cdc-5afec18b9b4c None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:50.393722 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20030f32-8a42-42d0-8cdc-5afec18b9b4c None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:50.394019 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20030f32-8a42-42d0-8cdc-5afec18b9b4c None admin] PATH_INFO: `/v3/projects/513641c037c84c54b779b5ca0684f17c` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:50.395205 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20030f32-8a42-42d0-8cdc-5afec18b9b4c None admin] RBAC: Authorizing `identity:get_project(project_id=513641c037c84c54b779b5ca0684f17c)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:50.396648 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20030f32-8a42-42d0-8cdc-5afec18b9b4c None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:50.399179 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 175/346] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 15 19:14:50 2022] GET /identity/v3/projects/513641c037c84c54b779b5ca0684f17c => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:50.400403 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d83fc9-a12d-4e2a-8831-e3583dbd3d4f None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:50.401084 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d83fc9-a12d-4e2a-8831-e3583dbd3d4f None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:50.401084 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d83fc9-a12d-4e2a-8831-e3583dbd3d4f None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:50.405842 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 176/347] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:14:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:14:50.405842 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d83fc9-a12d-4e2a-8831-e3583dbd3d4f None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:50.427779 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:50.446173 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5272de-a0ac-4473-8771-7918b3a53875 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:50.486300 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de347c0-05ed-4f43-8105-1dc9813b610b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:50.487119 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de347c0-05ed-4f43-8105-1dc9813b610b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:50.487404 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de347c0-05ed-4f43-8105-1dc9813b610b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:50.487686 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de347c0-05ed-4f43-8105-1dc9813b610b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:50.507830 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de347c0-05ed-4f43-8105-1dc9813b610b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:14:50.518492 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-1de347c0-05ed-4f43-8105-1dc9813b610b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 19:14:50.518492 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:14:50.518492 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:14:50.518492 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:14:50.518492 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:14:50.518492 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:14:50.518492 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:14:50.518492 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:14:50.518492 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:14:50.518492 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:14:50.518492 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:14:50.518492 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:14:50.518492 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:14:50.518492 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:14:50.518492 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:14:50.518492 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:14:50.519546 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:14:50.519546 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:14:50.519546 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:14:50.519546 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:14:50.519546 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:14:50.519546 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:14:50.519546 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:14:50.519546 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:14:50.519546 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:14:50.519546 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:14:50.519546 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:14:50.519546 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:14:50.519546 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:14:50.519546 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:14:50.519546 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:14:50.519546 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:14:50.520370 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:14:50.520370 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:14:50.520370 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:14:50.520370 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:14:50.520370 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:14:50.520370 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:14:50.520370 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:14:50.520370 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:14:50.520370 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:14:50.520370 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:14:50.520370 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 19:14:50.520370 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:14:50.520370 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 172/348] 10.222.0.33 () {56 vars in 1114 bytes} [Fri Jul 15 19:14:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:14:50.521054 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3878df-ee2d-48b2-b458-25b66cbb100c None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:50.521807 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3878df-ee2d-48b2-b458-25b66cbb100c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:50.522168 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3878df-ee2d-48b2-b458-25b66cbb100c None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:50.524646 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 173/349] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:14:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:14:50.525442 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3878df-ee2d-48b2-b458-25b66cbb100c None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:50.544094 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d83fc9-a12d-4e2a-8831-e3583dbd3d4f None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:50.586674 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10be38da-006c-4b8b-81b5-058f0637dc38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:50.587332 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10be38da-006c-4b8b-81b5-058f0637dc38 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:50.587664 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10be38da-006c-4b8b-81b5-058f0637dc38 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:50.587939 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10be38da-006c-4b8b-81b5-058f0637dc38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:50.658205 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:50.665851 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3878df-ee2d-48b2-b458-25b66cbb100c None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:50.669849 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10be38da-006c-4b8b-81b5-058f0637dc38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:50.669849 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10be38da-006c-4b8b-81b5-058f0637dc38 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:50.712334 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294b1fae-6c80-4114-99e8-916c68b53db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:50.713605 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294b1fae-6c80-4114-99e8-916c68b53db3 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:50.715781 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294b1fae-6c80-4114-99e8-916c68b53db3 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:50.715781 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294b1fae-6c80-4114-99e8-916c68b53db3 None admin] PATH_INFO: `/v3/groups/3d72e450754f4225bc7bc70c63a47fde` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:50.723809 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294b1fae-6c80-4114-99e8-916c68b53db3 None admin] RBAC: Authorizing `identity:get_group(group_id=3d72e450754f4225bc7bc70c63a47fde)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:50.725457 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294b1fae-6c80-4114-99e8-916c68b53db3 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:50.727385 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 174/350] 10.222.0.33 () {56 vars in 1180 bytes} [Fri Jul 15 19:14:50 2022] GET /identity/v3/groups/3d72e450754f4225bc7bc70c63a47fde => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:50.728493 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25d6dd5-4178-4e84-8e79-7fe8389af621 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:50.728983 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25d6dd5-4178-4e84-8e79-7fe8389af621 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:50.729098 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25d6dd5-4178-4e84-8e79-7fe8389af621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:50.748424 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-f25d6dd5-4178-4e84-8e79-7fe8389af621 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:14:50.749609 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-f25d6dd5-4178-4e84-8e79-7fe8389af621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:14:50.761704 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:50.812254 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 177/351] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 15 19:14:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 19:14:50.816432 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10be38da-006c-4b8b-81b5-058f0637dc38 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:50.840960 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 175/352] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:14:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:14:50.841321 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e79d08d-382e-44dd-8ac0-0c3e06ca5d63 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:50.841601 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e79d08d-382e-44dd-8ac0-0c3e06ca5d63 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:50.841918 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e79d08d-382e-44dd-8ac0-0c3e06ca5d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:50.869350 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-7e79d08d-382e-44dd-8ac0-0c3e06ca5d63 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:14:50.870600 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-7e79d08d-382e-44dd-8ac0-0c3e06ca5d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:14:50.931100 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:50.939249 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10be38da-006c-4b8b-81b5-058f0637dc38 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:50.951634 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 176/353] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:14:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:14:50.953508 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e79d08d-382e-44dd-8ac0-0c3e06ca5d63 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:50.987302 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7695b21d-b257-4bcd-805c-f94d70b4c0fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:50.987977 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7695b21d-b257-4bcd-805c-f94d70b4c0fa None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:50.988260 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7695b21d-b257-4bcd-805c-f94d70b4c0fa None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:50.988535 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7695b21d-b257-4bcd-805c-f94d70b4c0fa None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:50.989291 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7695b21d-b257-4bcd-805c-f94d70b4c0fa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:50.989604 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7695b21d-b257-4bcd-805c-f94d70b4c0fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:50.991073 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7695b21d-b257-4bcd-805c-f94d70b4c0fa None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:51.001610 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 178/354] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 15 19:14:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:51.002318 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7695b21d-b257-4bcd-805c-f94d70b4c0fa None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:51.035435 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:51.069131 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e79d08d-382e-44dd-8ac0-0c3e06ca5d63 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:51.110791 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d164ad7-d21a-479f-9400-e7bd1d6a8a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:51.111426 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d164ad7-d21a-479f-9400-e7bd1d6a8a2b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:51.111712 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d164ad7-d21a-479f-9400-e7bd1d6a8a2b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:51.111984 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d164ad7-d21a-479f-9400-e7bd1d6a8a2b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:51.112829 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d164ad7-d21a-479f-9400-e7bd1d6a8a2b None admin] RBAC: Adding query filter params (group.id=3d72e450754f4225bc7bc70c63a47fde, role.id=28763c6e70094116b3a4b2895895fa58, scope.project.id=513641c037c84c54b779b5ca0684f17c) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:51.113139 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d164ad7-d21a-479f-9400-e7bd1d6a8a2b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:51.114635 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d164ad7-d21a-479f-9400-e7bd1d6a8a2b None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:51.122103 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 177/355] 10.222.0.33 () {56 vars in 1376 bytes} [Fri Jul 15 19:14:50 2022] GET /identity/v3/role_assignments?group.id=3d72e450754f4225bc7bc70c63a47fde&scope.project.id=513641c037c84c54b779b5ca0684f17c&role.id=28763c6e70094116b3a4b2895895fa58 => generated 260 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:51.123531 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d510289-794d-4196-a45e-000186051118 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:51.123820 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d510289-794d-4196-a45e-000186051118 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:51.124091 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d510289-794d-4196-a45e-000186051118 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:51.125466 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 178/356] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:14:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:14:51.126063 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d510289-794d-4196-a45e-000186051118 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:51.131931 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7695b21d-b257-4bcd-805c-f94d70b4c0fa None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:51.177868 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d7fc71-d24a-4704-a891-90e264a83e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:51.178798 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d7fc71-d24a-4704-a891-90e264a83e82 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:51.179115 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d7fc71-d24a-4704-a891-90e264a83e82 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:51.179395 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d7fc71-d24a-4704-a891-90e264a83e82 None admin] PATH_INFO: `/v3/projects/12e6f4bc187942989f83aeb1902a3036` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:51.180656 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d7fc71-d24a-4704-a891-90e264a83e82 None admin] RBAC: Authorizing `identity:get_project(project_id=12e6f4bc187942989f83aeb1902a3036)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:51.182246 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d7fc71-d24a-4704-a891-90e264a83e82 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:51.184220 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 179/357] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 15 19:14:51 2022] GET /identity/v3/projects/12e6f4bc187942989f83aeb1902a3036 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:51.184837 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d7fc71-d24a-4704-a891-90e264a83e82 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:51.220898 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:51.268865 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d510289-794d-4196-a45e-000186051118 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:51.317614 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebee7cc4-45c2-45e0-8010-a944f5b175ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:51.318460 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee7cc4-45c2-45e0-8010-a944f5b175ec None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:51.319102 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee7cc4-45c2-45e0-8010-a944f5b175ec None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:51.319384 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee7cc4-45c2-45e0-8010-a944f5b175ec None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:51.327820 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:51.336974 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d7fc71-d24a-4704-a891-90e264a83e82 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:51.338675 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebee7cc4-45c2-45e0-8010-a944f5b175ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:14:51.346802 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-ebee7cc4-45c2-45e0-8010-a944f5b175ec None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 19:14:51.346802 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:14:51.346802 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:14:51.346802 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:14:51.346802 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:14:51.346802 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:14:51.346802 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:14:51.346802 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:14:51.346802 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:14:51.346802 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:14:51.346802 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:14:51.346802 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:14:51.346802 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:14:51.346802 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:14:51.346802 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:14:51.346802 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:14:51.347711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:14:51.347711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:14:51.347711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:14:51.347711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:14:51.347711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:14:51.347711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:14:51.347711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:14:51.347711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:14:51.347711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:14:51.347711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:14:51.347711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:14:51.347711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:14:51.347711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:14:51.347711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:14:51.347711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:14:51.347711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:14:51.348516 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:14:51.348516 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:14:51.348516 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:14:51.348516 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:14:51.348516 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:14:51.348516 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:14:51.348516 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:14:51.348516 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:14:51.348516 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:14:51.348516 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:14:51.348516 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 19:14:51.348516 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:14:51.348516 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 179/358] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 15 19:14:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:14:51.349180 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebee7cc4-45c2-45e0-8010-a944f5b175ec None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:51.385804 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f01e16-a0d4-4e86-a253-fd1da6bc5970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:51.386865 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f01e16-a0d4-4e86-a253-fd1da6bc5970 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:51.387321 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f01e16-a0d4-4e86-a253-fd1da6bc5970 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:51.387782 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f01e16-a0d4-4e86-a253-fd1da6bc5970 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:51.406255 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f01e16-a0d4-4e86-a253-fd1da6bc5970 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:14:51.415594 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-d8f01e16-a0d4-4e86-a253-fd1da6bc5970 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 19:14:51.415594 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:14:51.415594 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:14:51.415594 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:14:51.415594 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:14:51.415594 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:14:51.415594 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:14:51.415594 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:14:51.415594 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:14:51.415594 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:14:51.415594 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:14:51.415594 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:14:51.415594 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:14:51.415594 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:14:51.415594 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:14:51.415594 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:14:51.416514 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:14:51.416514 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:14:51.416514 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:14:51.416514 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:14:51.416514 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:14:51.416514 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:14:51.416514 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:14:51.416514 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:14:51.416514 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:14:51.416514 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:14:51.416514 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:14:51.416514 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:14:51.416514 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:14:51.416514 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:14:51.416514 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:14:51.416514 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:14:51.417865 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:14:51.417865 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:14:51.417865 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:14:51.417865 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:14:51.417865 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:14:51.417865 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:14:51.417865 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:14:51.417865 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:14:51.417865 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:14:51.417865 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:14:51.417865 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 19:14:51.417865 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:14:51.417865 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 180/359] 10.222.0.33 () {56 vars in 1114 bytes} [Fri Jul 15 19:14:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:14:51.418537 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94dab24-b8e9-403d-9d56-46dc655fec15 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:51.418846 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94dab24-b8e9-403d-9d56-46dc655fec15 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:51.419015 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94dab24-b8e9-403d-9d56-46dc655fec15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:51.438377 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-a94dab24-b8e9-403d-9d56-46dc655fec15 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:14:51.439495 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-a94dab24-b8e9-403d-9d56-46dc655fec15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:14:51.483788 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebee7cc4-45c2-45e0-8010-a944f5b175ec None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:51.533860 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 181/360] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:14:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:14:51.534908 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94dab24-b8e9-403d-9d56-46dc655fec15 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:51.537807 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee2e611-e4e4-4abf-a30f-6429308114c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:51.539227 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee2e611-e4e4-4abf-a30f-6429308114c1 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:51.540641 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee2e611-e4e4-4abf-a30f-6429308114c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:51.541176 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee2e611-e4e4-4abf-a30f-6429308114c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:51.553532 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:51.641566 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:51.651208 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee2e611-e4e4-4abf-a30f-6429308114c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:51.652674 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee2e611-e4e4-4abf-a30f-6429308114c1 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:51.678852 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94dab24-b8e9-403d-9d56-46dc655fec15 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:51.734333 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e0f35a-b2c5-4b08-a92d-475bf520cac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:51.735494 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e0f35a-b2c5-4b08-a92d-475bf520cac8 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:51.735779 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e0f35a-b2c5-4b08-a92d-475bf520cac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:51.736054 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e0f35a-b2c5-4b08-a92d-475bf520cac8 None admin] PATH_INFO: `/v3/groups/3d72e450754f4225bc7bc70c63a47fde` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:51.737227 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e0f35a-b2c5-4b08-a92d-475bf520cac8 None admin] RBAC: Authorizing `identity:get_group(group_id=3d72e450754f4225bc7bc70c63a47fde)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:51.738723 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e0f35a-b2c5-4b08-a92d-475bf520cac8 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:51.743507 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 182/361] 10.222.0.33 () {56 vars in 1180 bytes} [Fri Jul 15 19:14:51 2022] GET /identity/v3/groups/3d72e450754f4225bc7bc70c63a47fde => generated 226 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:51.743659 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e0f35a-b2c5-4b08-a92d-475bf520cac8 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:51.776691 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 180/362] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 15 19:14:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 19:14:51.782085 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee2e611-e4e4-4abf-a30f-6429308114c1 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:51.848874 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:51.870583 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e0f35a-b2c5-4b08-a92d-475bf520cac8 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:51.906250 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:51.915405 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee2e611-e4e4-4abf-a30f-6429308114c1 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:51.921927 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40efd36f-7b04-413c-ba78-f64fb1f28d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:51.922681 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40efd36f-7b04-413c-ba78-f64fb1f28d3e None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:51.923029 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40efd36f-7b04-413c-ba78-f64fb1f28d3e None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:51.923321 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40efd36f-7b04-413c-ba78-f64fb1f28d3e None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:51.924074 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40efd36f-7b04-413c-ba78-f64fb1f28d3e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:51.924375 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40efd36f-7b04-413c-ba78-f64fb1f28d3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:51.925744 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40efd36f-7b04-413c-ba78-f64fb1f28d3e None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:51.938413 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 183/363] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jul 15 19:14:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:51.944115 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40efd36f-7b04-413c-ba78-f64fb1f28d3e None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:51.961867 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cfe383-1c27-4412-95ec-23ba8f25900c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:51.962855 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cfe383-1c27-4412-95ec-23ba8f25900c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:51.963266 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cfe383-1c27-4412-95ec-23ba8f25900c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:51.963680 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cfe383-1c27-4412-95ec-23ba8f25900c None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:51.964635 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cfe383-1c27-4412-95ec-23ba8f25900c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:51.965412 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cfe383-1c27-4412-95ec-23ba8f25900c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:51.967265 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cfe383-1c27-4412-95ec-23ba8f25900c None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:51.978610 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 181/364] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 15 19:14:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:51.979953 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cfe383-1c27-4412-95ec-23ba8f25900c None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:52.090643 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40efd36f-7b04-413c-ba78-f64fb1f28d3e None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:52.117999 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cfe383-1c27-4412-95ec-23ba8f25900c None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:52.134214 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d6104e-f049-4bc4-ad1d-cc4e24bc5ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:52.135011 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d6104e-f049-4bc4-ad1d-cc4e24bc5ebf None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:52.135297 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d6104e-f049-4bc4-ad1d-cc4e24bc5ebf None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:52.135613 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d6104e-f049-4bc4-ad1d-cc4e24bc5ebf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:52.152027 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d6104e-f049-4bc4-ad1d-cc4e24bc5ebf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:14:52.159135 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19366ec1-5b70-4cfd-b099-658439513602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:52.159762 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-12d6104e-f049-4bc4-ad1d-cc4e24bc5ebf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:14:52.159762 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:14:52.159762 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:14:52.159762 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:14:52.159762 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:14:52.159762 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:14:52.159762 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:14:52.159762 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:14:52.159762 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:14:52.159762 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:14:52.159762 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:14:52.159762 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:14:52.159762 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:14:52.159762 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:14:52.159762 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:14:52.159762 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:14:52.160641 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:14:52.160641 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:14:52.160641 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:14:52.160641 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:14:52.160641 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:14:52.160641 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:14:52.160641 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:14:52.160641 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:14:52.160641 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:14:52.160641 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:14:52.160641 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:14:52.160641 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:14:52.160641 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:14:52.160641 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:14:52.160641 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:14:52.160641 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:14:52.161425 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:14:52.161425 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:14:52.161425 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:14:52.161425 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:14:52.161425 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:14:52.161425 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:14:52.161425 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:14:52.161425 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:14:52.161425 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:14:52.161425 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:14:52.161425 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:14:52.161425 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:14:52.162090 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19366ec1-5b70-4cfd-b099-658439513602 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:52.162090 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19366ec1-5b70-4cfd-b099-658439513602 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:52.162090 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19366ec1-5b70-4cfd-b099-658439513602 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:52.162090 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19366ec1-5b70-4cfd-b099-658439513602 None admin] RBAC: Adding query filter params (group.id=3d72e450754f4225bc7bc70c63a47fde, role.id=28763c6e70094116b3a4b2895895fa58, scope.project.id=12e6f4bc187942989f83aeb1902a3036) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:52.162090 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19366ec1-5b70-4cfd-b099-658439513602 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:52.163042 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19366ec1-5b70-4cfd-b099-658439513602 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:52.165934 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 184/365] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jul 15 19:14:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:14:52.167053 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d6104e-f049-4bc4-ad1d-cc4e24bc5ebf None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:52.170366 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 182/366] 10.222.0.33 () {56 vars in 1376 bytes} [Fri Jul 15 19:14:51 2022] GET /identity/v3/role_assignments?group.id=3d72e450754f4225bc7bc70c63a47fde&scope.project.id=12e6f4bc187942989f83aeb1902a3036&role.id=28763c6e70094116b3a4b2895895fa58 => generated 260 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:52.171287 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19366ec1-5b70-4cfd-b099-658439513602 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:52.172110 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:52.212468 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:52.307605 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d6104e-f049-4bc4-ad1d-cc4e24bc5ebf None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:52.348643 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71649660-5873-4b83-b09e-2f55ea7f1ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:52.349928 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71649660-5873-4b83-b09e-2f55ea7f1ff5 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:52.350433 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71649660-5873-4b83-b09e-2f55ea7f1ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:52.350974 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71649660-5873-4b83-b09e-2f55ea7f1ff5 None admin] PATH_INFO: `/v3/projects/513641c037c84c54b779b5ca0684f17c` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:52.352516 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71649660-5873-4b83-b09e-2f55ea7f1ff5 None admin] RBAC: Authorizing `identity:get_project(project_id=513641c037c84c54b779b5ca0684f17c)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:52.354251 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71649660-5873-4b83-b09e-2f55ea7f1ff5 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:52.356648 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 185/367] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 15 19:14:52 2022] GET /identity/v3/projects/513641c037c84c54b779b5ca0684f17c => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:52.357859 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71649660-5873-4b83-b09e-2f55ea7f1ff5 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:52.358194 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19366ec1-5b70-4cfd-b099-658439513602 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:52.406528 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92ec0f7-3b4c-4303-b865-af3042cf03b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:52.407055 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92ec0f7-3b4c-4303-b865-af3042cf03b7 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:52.407333 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92ec0f7-3b4c-4303-b865-af3042cf03b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:52.407602 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92ec0f7-3b4c-4303-b865-af3042cf03b7 None admin] PATH_INFO: `/v3/users/6ecf9665e7f1468ba57248f067821b3e` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:52.409718 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92ec0f7-3b4c-4303-b865-af3042cf03b7 None admin] RBAC: Authorizing `identity:get_user(user_id=6ecf9665e7f1468ba57248f067821b3e)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:52.411205 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92ec0f7-3b4c-4303-b865-af3042cf03b7 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:52.413294 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 183/368] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 15 19:14:52 2022] GET /identity/v3/users/6ecf9665e7f1468ba57248f067821b3e => generated 293 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:52.418267 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92ec0f7-3b4c-4303-b865-af3042cf03b7 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:52.449870 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:52.490064 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71649660-5873-4b83-b09e-2f55ea7f1ff5 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:52.528477 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:52.535980 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c47228e-711a-4f28-8965-245cdaa01132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:52.536905 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c47228e-711a-4f28-8965-245cdaa01132 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:52.537345 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c47228e-711a-4f28-8965-245cdaa01132 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:52.537911 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c47228e-711a-4f28-8965-245cdaa01132 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:52.539048 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92ec0f7-3b4c-4303-b865-af3042cf03b7 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:52.578707 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0efb92-9991-449f-aa80-9067ba8fa839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:52.579689 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0efb92-9991-449f-aa80-9067ba8fa839 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:52.580153 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0efb92-9991-449f-aa80-9067ba8fa839 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:52.580615 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0efb92-9991-449f-aa80-9067ba8fa839 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:52.581537 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0efb92-9991-449f-aa80-9067ba8fa839 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:52.582051 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0efb92-9991-449f-aa80-9067ba8fa839 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:52.583604 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0efb92-9991-449f-aa80-9067ba8fa839 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:52.594941 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 184/369] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 15 19:14:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:52.596145 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0efb92-9991-449f-aa80-9067ba8fa839 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:52.630725 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c47228e-711a-4f28-8965-245cdaa01132 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:52.633195 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c47228e-711a-4f28-8965-245cdaa01132 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:52.716421 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0efb92-9991-449f-aa80-9067ba8fa839 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:52.733952 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:52.757118 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04acb6a7-40b0-4c6b-a34a-abd8e4016921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:52.757993 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04acb6a7-40b0-4c6b-a34a-abd8e4016921 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:52.758269 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04acb6a7-40b0-4c6b-a34a-abd8e4016921 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:52.758542 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04acb6a7-40b0-4c6b-a34a-abd8e4016921 None admin] PATH_INFO: `/v3/groups/3d72e450754f4225bc7bc70c63a47fde` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:52.759729 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04acb6a7-40b0-4c6b-a34a-abd8e4016921 None admin] RBAC: Authorizing `identity:get_group(group_id=3d72e450754f4225bc7bc70c63a47fde)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:52.761632 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04acb6a7-40b0-4c6b-a34a-abd8e4016921 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:52.764118 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 185/370] 10.222.0.33 () {56 vars in 1180 bytes} [Fri Jul 15 19:14:52 2022] GET /identity/v3/groups/3d72e450754f4225bc7bc70c63a47fde => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:52.765408 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04acb6a7-40b0-4c6b-a34a-abd8e4016921 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:52.779113 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 186/371] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 15 19:14:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 19:14:52.780791 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c47228e-711a-4f28-8965-245cdaa01132 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:52.794348 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:52.880507 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04acb6a7-40b0-4c6b-a34a-abd8e4016921 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:52.920992 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5528c902-5e6c-430e-9d3a-3b299a1bda6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:52.921561 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5528c902-5e6c-430e-9d3a-3b299a1bda6e None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:52.921866 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5528c902-5e6c-430e-9d3a-3b299a1bda6e None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:52.922139 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5528c902-5e6c-430e-9d3a-3b299a1bda6e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:52.922963 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5528c902-5e6c-430e-9d3a-3b299a1bda6e None admin] RBAC: Adding query filter params (role.id=2f62da1ebb2b443eabf21e910a215048, user.id=6ecf9665e7f1468ba57248f067821b3e, scope.system=all) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:52.923268 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5528c902-5e6c-430e-9d3a-3b299a1bda6e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:52.924635 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c47228e-711a-4f28-8965-245cdaa01132 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:52.924755 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5528c902-5e6c-430e-9d3a-3b299a1bda6e None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:52.946089 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 186/372] 10.222.0.33 () {56 vars in 1308 bytes} [Fri Jul 15 19:14:52 2022] GET /identity/v3/role_assignments?user.id=6ecf9665e7f1468ba57248f067821b3e&scope.system=all&role.id=2f62da1ebb2b443eabf21e910a215048 => generated 510 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:52.947327 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5528c902-5e6c-430e-9d3a-3b299a1bda6e None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:52.969326 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f66fa7-5713-4653-ad11-5d9694f7e2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:52.970404 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f66fa7-5713-4653-ad11-5d9694f7e2dd None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:52.970893 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f66fa7-5713-4653-ad11-5d9694f7e2dd None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:52.971432 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f66fa7-5713-4653-ad11-5d9694f7e2dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:53.027034 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:53.064135 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f66fa7-5713-4653-ad11-5d9694f7e2dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:53.066037 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f66fa7-5713-4653-ad11-5d9694f7e2dd None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:53.162240 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:53.191195 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 187/373] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 15 19:14:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 19:14:53.192722 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f66fa7-5713-4653-ad11-5d9694f7e2dd None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:53.210179 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5528c902-5e6c-430e-9d3a-3b299a1bda6e None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:53.250347 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e2e780-1891-4e1b-89ba-8187f31e8c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:53.251009 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e2e780-1891-4e1b-89ba-8187f31e8c18 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:53.251291 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e2e780-1891-4e1b-89ba-8187f31e8c18 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:53.251567 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e2e780-1891-4e1b-89ba-8187f31e8c18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:53.252579 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e2e780-1891-4e1b-89ba-8187f31e8c18 None admin] RBAC: Adding query filter params (group.id=3d72e450754f4225bc7bc70c63a47fde, role.id=e97d26621445466d9550261c608db9c5, scope.project.id=513641c037c84c54b779b5ca0684f17c) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:53.252920 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e2e780-1891-4e1b-89ba-8187f31e8c18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:53.254419 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e2e780-1891-4e1b-89ba-8187f31e8c18 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:53.262855 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 187/374] 10.222.0.33 () {56 vars in 1376 bytes} [Fri Jul 15 19:14:52 2022] GET /identity/v3/role_assignments?group.id=3d72e450754f4225bc7bc70c63a47fde&scope.project.id=513641c037c84c54b779b5ca0684f17c&role.id=e97d26621445466d9550261c608db9c5 => generated 260 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:53.263694 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e2e780-1891-4e1b-89ba-8187f31e8c18 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:53.286431 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:53.326389 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f66fa7-5713-4653-ad11-5d9694f7e2dd None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:53.383080 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702ed0d3-5427-47e9-8718-4b5abcab3b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:53.383986 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ed0d3-5427-47e9-8718-4b5abcab3b99 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:53.384486 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ed0d3-5427-47e9-8718-4b5abcab3b99 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:53.384903 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ed0d3-5427-47e9-8718-4b5abcab3b99 None admin] PATH_INFO: `/v3/projects/12e6f4bc187942989f83aeb1902a3036` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:53.386348 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702ed0d3-5427-47e9-8718-4b5abcab3b99 None admin] RBAC: Authorizing `identity:get_project(project_id=12e6f4bc187942989f83aeb1902a3036)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:53.387894 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702ed0d3-5427-47e9-8718-4b5abcab3b99 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:53.389939 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 188/375] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 15 19:14:53 2022] GET /identity/v3/projects/12e6f4bc187942989f83aeb1902a3036 => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:53.394054 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d65f17-c59a-47eb-8798-24c3ee17b1dd None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:53.394357 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d65f17-c59a-47eb-8798-24c3ee17b1dd None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:53.394688 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d65f17-c59a-47eb-8798-24c3ee17b1dd None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:53.396349 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 189/376] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:14:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:14:53.397090 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d65f17-c59a-47eb-8798-24c3ee17b1dd None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:53.423240 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e2e780-1891-4e1b-89ba-8187f31e8c18 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:53.472372 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4f3447-8676-4dae-a1c0-bee1ff789add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:53.473290 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f3447-8676-4dae-a1c0-bee1ff789add None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:53.473576 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f3447-8676-4dae-a1c0-bee1ff789add None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:53.473894 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f3447-8676-4dae-a1c0-bee1ff789add None admin] PATH_INFO: `/v3/projects/392b3b5584be451c8c287eb751b4d7de` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:53.475126 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4f3447-8676-4dae-a1c0-bee1ff789add None admin] RBAC: Authorizing `identity:get_project(project_id=392b3b5584be451c8c287eb751b4d7de)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:53.477222 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4f3447-8676-4dae-a1c0-bee1ff789add None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:53.484386 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 188/377] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 15 19:14:53 2022] GET /identity/v3/projects/392b3b5584be451c8c287eb751b4d7de => generated 344 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:53.485713 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e642a1-974a-4ef1-8913-98cbc23bda6d None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:53.486053 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e642a1-974a-4ef1-8913-98cbc23bda6d None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:53.486331 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e642a1-974a-4ef1-8913-98cbc23bda6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:53.504178 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-49e642a1-974a-4ef1-8913-98cbc23bda6d None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:14:53.505414 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-49e642a1-974a-4ef1-8913-98cbc23bda6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:14:53.509830 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:53.547970 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d65f17-c59a-47eb-8798-24c3ee17b1dd None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:53.586438 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 189/378] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:14:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:14:53.588400 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e642a1-974a-4ef1-8913-98cbc23bda6d None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:53.592680 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587c39d4-fb38-488e-a1b9-491dedb53eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:53.593169 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c39d4-fb38-488e-a1b9-491dedb53eae None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:53.593448 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c39d4-fb38-488e-a1b9-491dedb53eae None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:53.593788 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c39d4-fb38-488e-a1b9-491dedb53eae None admin] PATH_INFO: `/v3/groups/3d72e450754f4225bc7bc70c63a47fde` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:53.595090 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587c39d4-fb38-488e-a1b9-491dedb53eae None admin] RBAC: Authorizing `identity:get_group(group_id=3d72e450754f4225bc7bc70c63a47fde)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:53.596661 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587c39d4-fb38-488e-a1b9-491dedb53eae None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:53.603989 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 190/379] 10.222.0.33 () {56 vars in 1180 bytes} [Fri Jul 15 19:14:53 2022] GET /identity/v3/groups/3d72e450754f4225bc7bc70c63a47fde => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:53.605234 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587c39d4-fb38-488e-a1b9-491dedb53eae None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:53.618118 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:53.725313 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587c39d4-fb38-488e-a1b9-491dedb53eae None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:53.730615 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e642a1-974a-4ef1-8913-98cbc23bda6d None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:53.768212 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d28f97-f562-4863-a971-154fd38f6a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:53.768874 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d28f97-f562-4863-a971-154fd38f6a79 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:53.769153 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d28f97-f562-4863-a971-154fd38f6a79 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:53.769424 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d28f97-f562-4863-a971-154fd38f6a79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:53.770296 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d28f97-f562-4863-a971-154fd38f6a79 None admin] RBAC: Adding query filter params (group.id=3d72e450754f4225bc7bc70c63a47fde, role.id=e97d26621445466d9550261c608db9c5, scope.project.id=12e6f4bc187942989f83aeb1902a3036) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:53.770613 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d28f97-f562-4863-a971-154fd38f6a79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:53.772039 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d28f97-f562-4863-a971-154fd38f6a79 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:53.783237 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c6019d-b56a-4622-b8ad-0a11fc878b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:53.784443 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c6019d-b56a-4622-b8ad-0a11fc878b52 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:53.784945 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c6019d-b56a-4622-b8ad-0a11fc878b52 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:53.785421 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c6019d-b56a-4622-b8ad-0a11fc878b52 None admin] PATH_INFO: `/v3/groups/f0d021f243a44e049c331ebf4f87cbab` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:53.786288 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 191/380] 10.222.0.33 () {56 vars in 1376 bytes} [Fri Jul 15 19:14:53 2022] GET /identity/v3/role_assignments?group.id=3d72e450754f4225bc7bc70c63a47fde&scope.project.id=12e6f4bc187942989f83aeb1902a3036&role.id=e97d26621445466d9550261c608db9c5 => generated 260 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:53.789791 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d28f97-f562-4863-a971-154fd38f6a79 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:53.803535 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c6019d-b56a-4622-b8ad-0a11fc878b52 None admin] RBAC: Authorizing `identity:get_group(group_id=f0d021f243a44e049c331ebf4f87cbab)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:53.805438 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c6019d-b56a-4622-b8ad-0a11fc878b52 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:53.812437 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 190/381] 10.222.0.33 () {56 vars in 1180 bytes} [Fri Jul 15 19:14:53 2022] GET /identity/v3/groups/f0d021f243a44e049c331ebf4f87cbab => generated 229 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:53.813797 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c6019d-b56a-4622-b8ad-0a11fc878b52 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:53.823332 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:53.928607 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:53.982714 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d28f97-f562-4863-a971-154fd38f6a79 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:53.997624 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c6019d-b56a-4622-b8ad-0a11fc878b52 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:54.028379 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94a5c8e-0a94-41d9-b4c5-5729fb71a253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:54.029545 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94a5c8e-0a94-41d9-b4c5-5729fb71a253 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:54.030107 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94a5c8e-0a94-41d9-b4c5-5729fb71a253 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:54.030710 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94a5c8e-0a94-41d9-b4c5-5729fb71a253 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:54.036235 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a885b0d1-d599-4902-a13a-ec6bdc6905b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:54.036913 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a885b0d1-d599-4902-a13a-ec6bdc6905b6 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:54.037199 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a885b0d1-d599-4902-a13a-ec6bdc6905b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:54.037547 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a885b0d1-d599-4902-a13a-ec6bdc6905b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:54.038879 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a885b0d1-d599-4902-a13a-ec6bdc6905b6 None admin] RBAC: Adding query filter params (group.id=f0d021f243a44e049c331ebf4f87cbab, role.id=4318f86ba9f74cd4a0bef1c311276657, scope.project.id=392b3b5584be451c8c287eb751b4d7de) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:54.039186 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a885b0d1-d599-4902-a13a-ec6bdc6905b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:54.040722 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a885b0d1-d599-4902-a13a-ec6bdc6905b6 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:54.045055 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94a5c8e-0a94-41d9-b4c5-5729fb71a253 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:14:54.047010 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 191/382] 10.222.0.33 () {56 vars in 1376 bytes} [Fri Jul 15 19:14:53 2022] GET /identity/v3/role_assignments?group.id=f0d021f243a44e049c331ebf4f87cbab&scope.project.id=392b3b5584be451c8c287eb751b4d7de&role.id=4318f86ba9f74cd4a0bef1c311276657 => generated 260 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:54.053630 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-e94a5c8e-0a94-41d9-b4c5-5729fb71a253 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 19:14:54.053630 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:14:54.053630 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:14:54.053630 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:14:54.053630 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:14:54.053630 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:14:54.053630 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:14:54.053630 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:14:54.053630 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:14:54.053630 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:14:54.053630 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:14:54.053630 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:14:54.053630 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:14:54.053630 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:14:54.053630 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:14:54.053630 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:14:54.054662 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:14:54.054662 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:14:54.054662 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:14:54.054662 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:14:54.054662 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:14:54.054662 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:14:54.054662 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:14:54.054662 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:14:54.054662 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:14:54.054662 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:14:54.054662 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:14:54.054662 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:14:54.054662 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:14:54.054662 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:14:54.054662 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:14:54.054662 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:14:54.055470 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:14:54.055470 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:14:54.055470 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:14:54.055470 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:14:54.055470 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:14:54.055470 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:14:54.055470 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:14:54.055470 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:14:54.055470 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:14:54.055470 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:14:54.055470 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 19:14:54.055470 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:14:54.058455 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 192/383] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 15 19:14:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:14:54.064233 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a885b0d1-d599-4902-a13a-ec6bdc6905b6 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:54.146378 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:54.187540 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a885b0d1-d599-4902-a13a-ec6bdc6905b6 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:54.225062 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e4dec6-d941-49a6-a4ef-7fe3093d451b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:54.225830 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e4dec6-d941-49a6-a4ef-7fe3093d451b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:54.226122 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e4dec6-d941-49a6-a4ef-7fe3093d451b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:54.226399 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e4dec6-d941-49a6-a4ef-7fe3093d451b None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:54.227162 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e4dec6-d941-49a6-a4ef-7fe3093d451b None admin] RBAC: Adding query filter params (name=member) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:54.227470 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e4dec6-d941-49a6-a4ef-7fe3093d451b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:54.228848 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e4dec6-d941-49a6-a4ef-7fe3093d451b None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:54.238268 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 192/384] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jul 15 19:14:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:54.243916 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94a5c8e-0a94-41d9-b4c5-5729fb71a253 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:54.421127 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94a5c8e-0a94-41d9-b4c5-5729fb71a253 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:54.474818 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:54.480476 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de455e83-d42c-4444-ae02-7be7a18cee9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:54.481335 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de455e83-d42c-4444-ae02-7be7a18cee9e None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:54.481610 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de455e83-d42c-4444-ae02-7be7a18cee9e None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:54.481909 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de455e83-d42c-4444-ae02-7be7a18cee9e None admin] PATH_INFO: `/v3/groups/3d72e450754f4225bc7bc70c63a47fde` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:54.483079 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de455e83-d42c-4444-ae02-7be7a18cee9e None admin] RBAC: Authorizing `identity:get_group(group_id=3d72e450754f4225bc7bc70c63a47fde)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:54.484494 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de455e83-d42c-4444-ae02-7be7a18cee9e None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:54.486360 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 193/385] 10.222.0.33 () {56 vars in 1180 bytes} [Fri Jul 15 19:14:54 2022] GET /identity/v3/groups/3d72e450754f4225bc7bc70c63a47fde => generated 226 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:54.492130 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e4dec6-d941-49a6-a4ef-7fe3093d451b None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:54.589754 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7116a3c-6bc5-4992-a764-1cae5a035257 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:54.590115 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7116a3c-6bc5-4992-a764-1cae5a035257 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:54.590375 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7116a3c-6bc5-4992-a764-1cae5a035257 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:54.593109 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 194/386] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:14:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 19:14:54.597425 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed3362c-4077-4d6e-bc07-452f48be3f3e None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:54.597713 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed3362c-4077-4d6e-bc07-452f48be3f3e None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:54.598016 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed3362c-4077-4d6e-bc07-452f48be3f3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:54.628108 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-8ed3362c-4077-4d6e-bc07-452f48be3f3e None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:14:54.629318 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-8ed3362c-4077-4d6e-bc07-452f48be3f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:14:54.649368 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e4dec6-d941-49a6-a4ef-7fe3093d451b None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:54.691220 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:54.698659 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2681ce-b3ec-4aac-9b08-e44e99d5caca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:54.700124 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2681ce-b3ec-4aac-9b08-e44e99d5caca None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:54.700589 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2681ce-b3ec-4aac-9b08-e44e99d5caca None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:54.701007 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2681ce-b3ec-4aac-9b08-e44e99d5caca None admin] PATH_INFO: `/v3/projects/513641c037c84c54b779b5ca0684f17c` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:54.702547 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2681ce-b3ec-4aac-9b08-e44e99d5caca None admin] RBAC: Authorizing `identity:get_project(project_id=513641c037c84c54b779b5ca0684f17c)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:54.704949 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2681ce-b3ec-4aac-9b08-e44e99d5caca None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:54.706765 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 193/387] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 15 19:14:54 2022] GET /identity/v3/projects/513641c037c84c54b779b5ca0684f17c => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:54.718834 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2681ce-b3ec-4aac-9b08-e44e99d5caca None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:54.753830 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 195/388] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:14:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 19:14:54.799296 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed3362c-4077-4d6e-bc07-452f48be3f3e None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:54.912391 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed3362c-4077-4d6e-bc07-452f48be3f3e None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:54.918607 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2681ce-b3ec-4aac-9b08-e44e99d5caca None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:54.970502 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:54.981111 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e893904-8170-49ff-91af-f2148a06ea50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:54.989487 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f98a25d-9ee7-41a8-ba28-ec4a091b3dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:54.989487 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f98a25d-9ee7-41a8-ba28-ec4a091b3dee None admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:54.989487 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f98a25d-9ee7-41a8-ba28-ec4a091b3dee None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:54.989487 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f98a25d-9ee7-41a8-ba28-ec4a091b3dee None admin] PATH_INFO: `/v3/projects/513641c037c84c54b779b5ca0684f17c/groups/3d72e450754f4225bc7bc70c63a47fde/roles/28763c6e70094116b3a4b2895895fa58` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:54.989839 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e893904-8170-49ff-91af-f2148a06ea50 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:54.989839 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e893904-8170-49ff-91af-f2148a06ea50 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:54.989839 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e893904-8170-49ff-91af-f2148a06ea50 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:54.990806 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f98a25d-9ee7-41a8-ba28-ec4a091b3dee None admin] RBAC: Authorizing `identity:create_grant(project_id=513641c037c84c54b779b5ca0684f17c, group_id=3d72e450754f4225bc7bc70c63a47fde, role_id=28763c6e70094116b3a4b2895895fa58)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:54.992680 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f98a25d-9ee7-41a8-ba28-ec4a091b3dee None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:55.002913 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e893904-8170-49ff-91af-f2148a06ea50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:14:55.011269 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-5e893904-8170-49ff-91af-f2148a06ea50 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 19:14:55.011269 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:14:55.011269 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:14:55.011269 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:14:55.011269 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:14:55.011269 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:14:55.011269 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:14:55.011269 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:14:55.011269 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:14:55.011269 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:14:55.011269 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:14:55.011269 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:14:55.011269 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:14:55.011269 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:14:55.011269 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:14:55.011269 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:14:55.012340 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:14:55.012340 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:14:55.012340 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:14:55.012340 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:14:55.012340 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:14:55.012340 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:14:55.012340 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:14:55.012340 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:14:55.012340 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:14:55.012340 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:14:55.012340 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:14:55.012340 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:14:55.012340 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:14:55.012340 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:14:55.012340 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:14:55.012340 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:14:55.013355 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:14:55.013355 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:14:55.013355 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:14:55.013355 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:14:55.013355 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:14:55.013355 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:14:55.013355 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:14:55.013355 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:14:55.013355 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:14:55.013355 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:14:55.013355 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 19:14:55.013355 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:14:55.015700 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 196/389] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 15 19:14:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:14:55.021816 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e893904-8170-49ff-91af-f2148a06ea50 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:55.091377 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 194/390] 10.222.0.33 () {58 vars in 1442 bytes} [Fri Jul 15 19:14:54 2022] PUT /identity/v3/projects/513641c037c84c54b779b5ca0684f17c/groups/3d72e450754f4225bc7bc70c63a47fde/roles/28763c6e70094116b3a4b2895895fa58 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:14:55.150846 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e893904-8170-49ff-91af-f2148a06ea50 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:55.205584 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ab5eb7-a897-41ce-82dc-6d62a41d66ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:55.206298 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ab5eb7-a897-41ce-82dc-6d62a41d66ad None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:55.206582 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ab5eb7-a897-41ce-82dc-6d62a41d66ad None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:55.206853 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ab5eb7-a897-41ce-82dc-6d62a41d66ad None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:55.207627 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ab5eb7-a897-41ce-82dc-6d62a41d66ad None admin] RBAC: Adding query filter params (name=member) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:55.207929 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ab5eb7-a897-41ce-82dc-6d62a41d66ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:55.209321 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ab5eb7-a897-41ce-82dc-6d62a41d66ad None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:55.226700 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 197/391] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jul 15 19:14:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:55.232900 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f98a25d-9ee7-41a8-ba28-ec4a091b3dee None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:55.327961 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:55.364433 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f98a25d-9ee7-41a8-ba28-ec4a091b3dee None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:55.401753 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdb2f98-4724-4fda-b3d5-021cfd18db4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:55.403482 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdb2f98-4724-4fda-b3d5-021cfd18db4d None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:55.404457 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdb2f98-4724-4fda-b3d5-021cfd18db4d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:55.405445 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdb2f98-4724-4fda-b3d5-021cfd18db4d None admin] PATH_INFO: `/v3/groups/3d72e450754f4225bc7bc70c63a47fde` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:55.407139 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdb2f98-4724-4fda-b3d5-021cfd18db4d None admin] RBAC: Authorizing `identity:get_group(group_id=3d72e450754f4225bc7bc70c63a47fde)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:55.408981 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdb2f98-4724-4fda-b3d5-021cfd18db4d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:55.414207 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 195/392] 10.222.0.33 () {56 vars in 1180 bytes} [Fri Jul 15 19:14:55 2022] GET /identity/v3/groups/3d72e450754f4225bc7bc70c63a47fde => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:55.417859 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ab5eb7-a897-41ce-82dc-6d62a41d66ad None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:55.473396 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:55.535838 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ab5eb7-a897-41ce-82dc-6d62a41d66ad None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:55.581359 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472fbcce-a69b-4e38-833b-f1eda38ef59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:55.582575 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472fbcce-a69b-4e38-833b-f1eda38ef59f None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:55.583078 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472fbcce-a69b-4e38-833b-f1eda38ef59f None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:55.583546 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472fbcce-a69b-4e38-833b-f1eda38ef59f None admin] PATH_INFO: `/v3/projects/12e6f4bc187942989f83aeb1902a3036` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:55.585011 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472fbcce-a69b-4e38-833b-f1eda38ef59f None admin] RBAC: Authorizing `identity:get_project(project_id=12e6f4bc187942989f83aeb1902a3036)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:55.586739 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472fbcce-a69b-4e38-833b-f1eda38ef59f None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:55.588709 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 198/393] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 15 19:14:55 2022] GET /identity/v3/projects/12e6f4bc187942989f83aeb1902a3036 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:14:55.594722 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdb2f98-4724-4fda-b3d5-021cfd18db4d None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:55.703449 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058de93f-4889-414f-a9fc-93ceec9d6741 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:55.703746 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058de93f-4889-414f-a9fc-93ceec9d6741 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:55.703995 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058de93f-4889-414f-a9fc-93ceec9d6741 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:55.705561 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 199/394] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:14:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 19:14:55.713072 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d724b6c-8ae9-4fec-8d7b-fd71511e8755 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:55.713673 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d724b6c-8ae9-4fec-8d7b-fd71511e8755 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:55.714200 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d724b6c-8ae9-4fec-8d7b-fd71511e8755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:55.736098 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-0d724b6c-8ae9-4fec-8d7b-fd71511e8755 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:14:55.738310 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-0d724b6c-8ae9-4fec-8d7b-fd71511e8755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:14:55.758727 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdb2f98-4724-4fda-b3d5-021cfd18db4d None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:55.829939 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 200/395] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:14:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 19:14:55.830477 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:55.845276 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5539c9-8677-4c5d-b931-90f7c63d9a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:55.846152 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5539c9-8677-4c5d-b931-90f7c63d9a62 None admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:55.846440 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5539c9-8677-4c5d-b931-90f7c63d9a62 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:55.846717 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5539c9-8677-4c5d-b931-90f7c63d9a62 None admin] PATH_INFO: `/v3/projects/12e6f4bc187942989f83aeb1902a3036/groups/3d72e450754f4225bc7bc70c63a47fde/roles/28763c6e70094116b3a4b2895895fa58` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:55.854564 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5539c9-8677-4c5d-b931-90f7c63d9a62 None admin] RBAC: Authorizing `identity:create_grant(project_id=12e6f4bc187942989f83aeb1902a3036, group_id=3d72e450754f4225bc7bc70c63a47fde, role_id=28763c6e70094116b3a4b2895895fa58)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:55.857612 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5539c9-8677-4c5d-b931-90f7c63d9a62 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:55.868442 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d724b6c-8ae9-4fec-8d7b-fd71511e8755 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:55.923799 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 196/396] 10.222.0.33 () {58 vars in 1442 bytes} [Fri Jul 15 19:14:55 2022] PUT /identity/v3/projects/12e6f4bc187942989f83aeb1902a3036/groups/3d72e450754f4225bc7bc70c63a47fde/roles/28763c6e70094116b3a4b2895895fa58 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:14:55.927155 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:55.981792 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d724b6c-8ae9-4fec-8d7b-fd71511e8755 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:56.018802 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e701b90b-a97e-40f7-9bf6-351ab4f6c90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:56.019640 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e701b90b-a97e-40f7-9bf6-351ab4f6c90b None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:56.019927 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e701b90b-a97e-40f7-9bf6-351ab4f6c90b None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:56.020209 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e701b90b-a97e-40f7-9bf6-351ab4f6c90b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:56.032993 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e701b90b-a97e-40f7-9bf6-351ab4f6c90b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:14:56.041883 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-e701b90b-a97e-40f7-9bf6-351ab4f6c90b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 19:14:56.041883 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:14:56.041883 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:14:56.041883 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:14:56.041883 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:14:56.041883 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:14:56.041883 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:14:56.041883 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:14:56.041883 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:14:56.041883 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:14:56.041883 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:14:56.041883 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:14:56.041883 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:14:56.041883 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:14:56.041883 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:14:56.041883 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:14:56.042810 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:14:56.042810 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:14:56.042810 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:14:56.042810 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:14:56.042810 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:14:56.042810 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:14:56.042810 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:14:56.042810 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:14:56.042810 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:14:56.042810 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:14:56.042810 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:14:56.042810 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:14:56.042810 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:14:56.042810 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:14:56.042810 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:14:56.042810 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:14:56.043672 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:14:56.043672 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:14:56.043672 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:14:56.043672 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:14:56.043672 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:14:56.043672 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:14:56.043672 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:14:56.043672 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:14:56.043672 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:14:56.043672 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:14:56.043672 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 19:14:56.043672 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:14:56.043672 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 201/397] 10.222.0.33 () {56 vars in 1114 bytes} [Fri Jul 15 19:14:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:14:56.049416 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5539c9-8677-4c5d-b931-90f7c63d9a62 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:56.186606 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5539c9-8677-4c5d-b931-90f7c63d9a62 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:56.224402 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d490ee-248a-46f3-b302-e193597184b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:56.225079 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d490ee-248a-46f3-b302-e193597184b9 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:56.225380 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d490ee-248a-46f3-b302-e193597184b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:56.226194 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d490ee-248a-46f3-b302-e193597184b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:56.226953 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d490ee-248a-46f3-b302-e193597184b9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:56.227260 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d490ee-248a-46f3-b302-e193597184b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:56.228918 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d490ee-248a-46f3-b302-e193597184b9 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:56.239039 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 197/398] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 15 19:14:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:56.244598 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e701b90b-a97e-40f7-9bf6-351ab4f6c90b None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:56.351684 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e701b90b-a97e-40f7-9bf6-351ab4f6c90b None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:56.363547 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:56.386589 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f017ff89-4c9a-474d-922c-6d8563f1b53b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:56.387455 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f017ff89-4c9a-474d-922c-6d8563f1b53b None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:56.387739 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f017ff89-4c9a-474d-922c-6d8563f1b53b None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:56.388025 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f017ff89-4c9a-474d-922c-6d8563f1b53b None admin] PATH_INFO: `/v3/groups/3d72e450754f4225bc7bc70c63a47fde` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:56.389222 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f017ff89-4c9a-474d-922c-6d8563f1b53b None admin] RBAC: Authorizing `identity:get_group(group_id=3d72e450754f4225bc7bc70c63a47fde)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:56.390760 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f017ff89-4c9a-474d-922c-6d8563f1b53b None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:56.392459 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 202/399] 10.222.0.33 () {56 vars in 1180 bytes} [Fri Jul 15 19:14:56 2022] GET /identity/v3/groups/3d72e450754f4225bc7bc70c63a47fde => generated 226 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:56.398792 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d490ee-248a-46f3-b302-e193597184b9 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:56.459489 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e82a4-ac98-4634-89db-50f9dc179f19 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:56.460583 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e82a4-ac98-4634-89db-50f9dc179f19 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:56.460658 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e82a4-ac98-4634-89db-50f9dc179f19 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:56.462206 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 203/400] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:14:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 19:14:56.470918 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:56.478901 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc06a00-2efb-4077-8f8d-18b96f5a360b None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:56.485133 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc06a00-2efb-4077-8f8d-18b96f5a360b None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:56.485133 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc06a00-2efb-4077-8f8d-18b96f5a360b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:56.502767 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-1cc06a00-2efb-4077-8f8d-18b96f5a360b None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:14:56.503974 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-1cc06a00-2efb-4077-8f8d-18b96f5a360b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:14:56.569175 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d490ee-248a-46f3-b302-e193597184b9 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:56.611827 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3192a641-5dcc-4331-8003-6b19764299f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:56.612849 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192a641-5dcc-4331-8003-6b19764299f8 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:56.613449 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192a641-5dcc-4331-8003-6b19764299f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:56.613967 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192a641-5dcc-4331-8003-6b19764299f8 None admin] PATH_INFO: `/v3/projects/513641c037c84c54b779b5ca0684f17c` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:56.615555 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3192a641-5dcc-4331-8003-6b19764299f8 None admin] RBAC: Authorizing `identity:get_project(project_id=513641c037c84c54b779b5ca0684f17c)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:56.617596 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3192a641-5dcc-4331-8003-6b19764299f8 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:56.619724 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 198/401] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 15 19:14:56 2022] GET /identity/v3/projects/513641c037c84c54b779b5ca0684f17c => generated 298 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:56.625199 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3192a641-5dcc-4331-8003-6b19764299f8 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:56.640138 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 204/402] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:14:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:14:56.677127 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc06a00-2efb-4077-8f8d-18b96f5a360b None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:56.801985 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3192a641-5dcc-4331-8003-6b19764299f8 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:56.835774 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:56.868557 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc06a00-2efb-4077-8f8d-18b96f5a360b None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:56.871746 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed47a422-fa61-45bb-8552-84f8ba8e5dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:56.872933 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed47a422-fa61-45bb-8552-84f8ba8e5dac None admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:56.873417 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed47a422-fa61-45bb-8552-84f8ba8e5dac None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:56.873923 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed47a422-fa61-45bb-8552-84f8ba8e5dac None admin] PATH_INFO: `/v3/projects/513641c037c84c54b779b5ca0684f17c/groups/3d72e450754f4225bc7bc70c63a47fde/roles/e97d26621445466d9550261c608db9c5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:56.877551 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed47a422-fa61-45bb-8552-84f8ba8e5dac None admin] RBAC: Authorizing `identity:create_grant(project_id=513641c037c84c54b779b5ca0684f17c, group_id=3d72e450754f4225bc7bc70c63a47fde, role_id=e97d26621445466d9550261c608db9c5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:56.879271 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed47a422-fa61-45bb-8552-84f8ba8e5dac None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:56.885349 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:56.910635 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fbaae5-a132-4987-beb8-663490bf0dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:56.911459 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fbaae5-a132-4987-beb8-663490bf0dbb None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:56.911736 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fbaae5-a132-4987-beb8-663490bf0dbb None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:56.912010 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fbaae5-a132-4987-beb8-663490bf0dbb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:56.922128 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fbaae5-a132-4987-beb8-663490bf0dbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:14:56.930696 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-40fbaae5-a132-4987-beb8-663490bf0dbb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 19:14:56.930696 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:14:56.930696 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:14:56.930696 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:14:56.930696 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:14:56.930696 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:14:56.930696 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:14:56.930696 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:14:56.930696 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:14:56.930696 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:14:56.930696 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:14:56.930696 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:14:56.930696 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:14:56.930696 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:14:56.930696 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:14:56.930696 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:14:56.931627 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:14:56.931627 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:14:56.931627 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:14:56.931627 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:14:56.931627 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:14:56.931627 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:14:56.931627 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:14:56.931627 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:14:56.931627 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:14:56.931627 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:14:56.931627 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:14:56.931627 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:14:56.931627 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:14:56.931627 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:14:56.931627 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:14:56.931627 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:14:56.932464 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:14:56.932464 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:14:56.932464 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:14:56.932464 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:14:56.932464 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:14:56.932464 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:14:56.932464 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:14:56.932464 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:14:56.932464 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:14:56.932464 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:14:56.932464 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 19:14:56.932464 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:14:56.932464 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 205/403] 10.222.0.33 () {56 vars in 1114 bytes} [Fri Jul 15 19:14:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:14:56.938109 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75943938-8898-4127-a817-936295985fa0 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:56.938109 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75943938-8898-4127-a817-936295985fa0 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:56.938109 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75943938-8898-4127-a817-936295985fa0 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:56.939222 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 206/404] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:14:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:14:56.940553 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75943938-8898-4127-a817-936295985fa0 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:56.970277 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 199/405] 10.222.0.33 () {58 vars in 1442 bytes} [Fri Jul 15 19:14:56 2022] PUT /identity/v3/projects/513641c037c84c54b779b5ca0684f17c/groups/3d72e450754f4225bc7bc70c63a47fde/roles/e97d26621445466d9550261c608db9c5 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:14:56.977795 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8831c937-ca34-42a8-ab5e-2297dcf38e79 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:56.978648 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8831c937-ca34-42a8-ab5e-2297dcf38e79 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:56.979160 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8831c937-ca34-42a8-ab5e-2297dcf38e79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:57.023864 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-8831c937-ca34-42a8-ab5e-2297dcf38e79 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:14:57.025450 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-8831c937-ca34-42a8-ab5e-2297dcf38e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:14:57.076669 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75943938-8898-4127-a817-936295985fa0 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:57.170540 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 200/406] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:14:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:14:57.171733 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8171949-921b-43cd-abbd-c66556e7d534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:57.172407 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8171949-921b-43cd-abbd-c66556e7d534 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:57.172683 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8171949-921b-43cd-abbd-c66556e7d534 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:57.172951 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8171949-921b-43cd-abbd-c66556e7d534 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:57.173702 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8171949-921b-43cd-abbd-c66556e7d534 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:57.174034 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8171949-921b-43cd-abbd-c66556e7d534 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:57.175411 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8171949-921b-43cd-abbd-c66556e7d534 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:57.186299 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 207/407] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 15 19:14:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:57.191804 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8831c937-ca34-42a8-ab5e-2297dcf38e79 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:57.200654 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8171949-921b-43cd-abbd-c66556e7d534 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:57.253401 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:57.324509 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8171949-921b-43cd-abbd-c66556e7d534 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:57.362745 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af71b9dd-26d5-4c99-8a66-b0c72f7c466b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:57.363474 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af71b9dd-26d5-4c99-8a66-b0c72f7c466b None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:57.363755 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af71b9dd-26d5-4c99-8a66-b0c72f7c466b None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:57.364028 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af71b9dd-26d5-4c99-8a66-b0c72f7c466b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:57.381155 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af71b9dd-26d5-4c99-8a66-b0c72f7c466b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:14:57.389144 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-af71b9dd-26d5-4c99-8a66-b0c72f7c466b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:14:57.389144 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:14:57.389144 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:14:57.389144 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:14:57.389144 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:14:57.389144 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:14:57.389144 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:14:57.389144 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:14:57.389144 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:14:57.389144 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:14:57.389144 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:14:57.389144 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:14:57.389144 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:14:57.389144 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:14:57.389144 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:14:57.389144 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:14:57.393451 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:14:57.393451 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:14:57.393451 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:14:57.393451 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:14:57.393451 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:14:57.393451 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:14:57.393451 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:14:57.393451 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:14:57.393451 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:14:57.393451 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:14:57.393451 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:14:57.393451 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:14:57.393451 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:14:57.393451 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:14:57.393451 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:14:57.393451 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:14:57.394305 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:14:57.394305 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:14:57.394305 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:14:57.394305 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:14:57.394305 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:14:57.394305 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:14:57.394305 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:14:57.394305 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:14:57.394305 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:14:57.394305 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:14:57.394305 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:14:57.394305 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:14:57.398777 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 208/408] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jul 15 19:14:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:14:57.401075 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8831c937-ca34-42a8-ab5e-2297dcf38e79 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:57.405719 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af71b9dd-26d5-4c99-8a66-b0c72f7c466b None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:57.413425 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:57.462244 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dba335b-edf5-4d5a-ae22-1446251c4a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:57.463117 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dba335b-edf5-4d5a-ae22-1446251c4a3a None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:57.463402 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dba335b-edf5-4d5a-ae22-1446251c4a3a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:57.463672 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dba335b-edf5-4d5a-ae22-1446251c4a3a None admin] PATH_INFO: `/v3/groups/3d72e450754f4225bc7bc70c63a47fde` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:57.465717 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dba335b-edf5-4d5a-ae22-1446251c4a3a None admin] RBAC: Authorizing `identity:get_group(group_id=3d72e450754f4225bc7bc70c63a47fde)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:57.467191 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dba335b-edf5-4d5a-ae22-1446251c4a3a None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:57.468999 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 201/409] 10.222.0.33 () {56 vars in 1180 bytes} [Fri Jul 15 19:14:57 2022] GET /identity/v3/groups/3d72e450754f4225bc7bc70c63a47fde => generated 226 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:57.475139 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dba335b-edf5-4d5a-ae22-1446251c4a3a None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:57.567251 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:57.577591 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af71b9dd-26d5-4c99-8a66-b0c72f7c466b None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:57.613321 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dba335b-edf5-4d5a-ae22-1446251c4a3a None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:57.614300 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00a09a6-ce39-4896-a244-e68f2d3613d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:57.615199 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00a09a6-ce39-4896-a244-e68f2d3613d6 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:57.615811 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00a09a6-ce39-4896-a244-e68f2d3613d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:57.616353 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00a09a6-ce39-4896-a244-e68f2d3613d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:57.617318 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00a09a6-ce39-4896-a244-e68f2d3613d6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:57.617852 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00a09a6-ce39-4896-a244-e68f2d3613d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:57.619454 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00a09a6-ce39-4896-a244-e68f2d3613d6 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:57.629050 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 209/410] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 15 19:14:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:57.630606 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e33b0-a88d-4d36-ac89-2db7747e2bf7 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:57.631111 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e33b0-a88d-4d36-ac89-2db7747e2bf7 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:57.631591 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e33b0-a88d-4d36-ac89-2db7747e2bf7 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:57.633903 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 210/411] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:14:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:14:57.638966 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19761fc0-ba83-4833-84a5-c5b8f1c72ef8 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:57.639495 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19761fc0-ba83-4833-84a5-c5b8f1c72ef8 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:57.639967 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19761fc0-ba83-4833-84a5-c5b8f1c72ef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:57.658036 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-19761fc0-ba83-4833-84a5-c5b8f1c72ef8 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:14:57.659497 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52170084-69fc-4cda-902c-d721f4f66205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:57.659859 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-19761fc0-ba83-4833-84a5-c5b8f1c72ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:14:57.660539 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52170084-69fc-4cda-902c-d721f4f66205 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:57.660882 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52170084-69fc-4cda-902c-d721f4f66205 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:57.661273 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52170084-69fc-4cda-902c-d721f4f66205 None admin] PATH_INFO: `/v3/projects/12e6f4bc187942989f83aeb1902a3036` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:57.662871 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52170084-69fc-4cda-902c-d721f4f66205 None admin] RBAC: Authorizing `identity:get_project(project_id=12e6f4bc187942989f83aeb1902a3036)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:57.665934 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52170084-69fc-4cda-902c-d721f4f66205 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:57.672612 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 202/412] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 15 19:14:57 2022] GET /identity/v3/projects/12e6f4bc187942989f83aeb1902a3036 => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:57.672612 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52170084-69fc-4cda-902c-d721f4f66205 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:57.732901 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:57.741859 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 211/413] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:14:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 19:14:57.742954 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19761fc0-ba83-4833-84a5-c5b8f1c72ef8 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:57.820634 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52170084-69fc-4cda-902c-d721f4f66205 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:57.860585 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0403622c-f4b1-4a45-aa84-284d467cb0b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:57.861755 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0403622c-f4b1-4a45-aa84-284d467cb0b1 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:57.862233 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0403622c-f4b1-4a45-aa84-284d467cb0b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:57.862703 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0403622c-f4b1-4a45-aa84-284d467cb0b1 None admin] PATH_INFO: `/v3/groups/f0d021f243a44e049c331ebf4f87cbab` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:57.864105 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0403622c-f4b1-4a45-aa84-284d467cb0b1 None admin] RBAC: Authorizing `identity:get_group(group_id=f0d021f243a44e049c331ebf4f87cbab)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:57.865695 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0403622c-f4b1-4a45-aa84-284d467cb0b1 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:57.870371 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:57.872208 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 203/414] 10.222.0.33 () {56 vars in 1180 bytes} [Fri Jul 15 19:14:57 2022] GET /identity/v3/groups/f0d021f243a44e049c331ebf4f87cbab => generated 229 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:57.873059 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0403622c-f4b1-4a45-aa84-284d467cb0b1 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:57.892890 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19761fc0-ba83-4833-84a5-c5b8f1c72ef8 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:57.930072 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4fd829-bde3-404f-9b42-8fc521c0d837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:57.931001 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4fd829-bde3-404f-9b42-8fc521c0d837 None admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:57.931332 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4fd829-bde3-404f-9b42-8fc521c0d837 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:57.931615 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4fd829-bde3-404f-9b42-8fc521c0d837 None admin] PATH_INFO: `/v3/projects/12e6f4bc187942989f83aeb1902a3036/groups/3d72e450754f4225bc7bc70c63a47fde/roles/e97d26621445466d9550261c608db9c5` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:57.933943 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4fd829-bde3-404f-9b42-8fc521c0d837 None admin] RBAC: Authorizing `identity:create_grant(project_id=12e6f4bc187942989f83aeb1902a3036, group_id=3d72e450754f4225bc7bc70c63a47fde, role_id=e97d26621445466d9550261c608db9c5)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:57.936286 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4fd829-bde3-404f-9b42-8fc521c0d837 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:57.993621 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0403622c-f4b1-4a45-aa84-284d467cb0b1 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:58.005799 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:58.007381 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 212/415] 10.222.0.33 () {58 vars in 1442 bytes} [Fri Jul 15 19:14:57 2022] PUT /identity/v3/projects/12e6f4bc187942989f83aeb1902a3036/groups/3d72e450754f4225bc7bc70c63a47fde/roles/e97d26621445466d9550261c608db9c5 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:14:58.009446 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4fd829-bde3-404f-9b42-8fc521c0d837 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:58.050002 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9075c61f-2c5d-4f87-b93a-6cb8c0969271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:58.050792 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9075c61f-2c5d-4f87-b93a-6cb8c0969271 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:58.051077 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9075c61f-2c5d-4f87-b93a-6cb8c0969271 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:58.051355 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9075c61f-2c5d-4f87-b93a-6cb8c0969271 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:58.074883 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9075c61f-2c5d-4f87-b93a-6cb8c0969271 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:14:58.090074 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-9075c61f-2c5d-4f87-b93a-6cb8c0969271 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 19:14:58.090074 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:14:58.090074 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:14:58.090074 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:14:58.090074 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:14:58.090074 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:14:58.090074 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:14:58.090074 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:14:58.090074 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:14:58.090074 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:14:58.090074 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:14:58.090074 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:14:58.090074 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:14:58.090074 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:14:58.090074 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:14:58.090074 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:14:58.091059 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:14:58.091059 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:14:58.091059 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:14:58.091059 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:14:58.091059 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:14:58.091059 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:14:58.091059 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:14:58.091059 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:14:58.091059 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:14:58.091059 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:14:58.091059 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:14:58.091059 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:14:58.091059 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:14:58.091059 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:14:58.091059 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:14:58.091059 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:14:58.092288 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:14:58.092288 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:14:58.092288 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:14:58.092288 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:14:58.092288 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:14:58.092288 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:14:58.092288 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:14:58.092288 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:14:58.092288 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:14:58.092288 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:14:58.092288 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 19:14:58.092288 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:14:58.092288 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 204/416] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 15 19:14:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:14:58.097397 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9075c61f-2c5d-4f87-b93a-6cb8c0969271 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:58.139960 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4fd829-bde3-404f-9b42-8fc521c0d837 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:58.186185 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdfbbca-235e-4ebf-a5d3-f6de099df483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:58.187519 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdfbbca-235e-4ebf-a5d3-f6de099df483 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:58.188005 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdfbbca-235e-4ebf-a5d3-f6de099df483 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:58.188579 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdfbbca-235e-4ebf-a5d3-f6de099df483 None admin] PATH_INFO: `/v3/projects/392b3b5584be451c8c287eb751b4d7de` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:58.190103 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdfbbca-235e-4ebf-a5d3-f6de099df483 None admin] RBAC: Authorizing `identity:get_project(project_id=392b3b5584be451c8c287eb751b4d7de)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:58.191797 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdfbbca-235e-4ebf-a5d3-f6de099df483 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:58.198959 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 213/417] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 15 19:14:58 2022] GET /identity/v3/projects/392b3b5584be451c8c287eb751b4d7de => generated 344 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:58.202185 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdfbbca-235e-4ebf-a5d3-f6de099df483 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:58.218158 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:58.286138 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9075c61f-2c5d-4f87-b93a-6cb8c0969271 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:58.343526 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ac6457-ea62-4676-a0b1-dc5415e02326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:58.344212 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ac6457-ea62-4676-a0b1-dc5415e02326 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:58.344493 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ac6457-ea62-4676-a0b1-dc5415e02326 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:58.344771 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ac6457-ea62-4676-a0b1-dc5415e02326 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:58.345536 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ac6457-ea62-4676-a0b1-dc5415e02326 None admin] RBAC: Adding query filter params (name=member) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:58.345872 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ac6457-ea62-4676-a0b1-dc5415e02326 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:58.347337 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ac6457-ea62-4676-a0b1-dc5415e02326 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:58.352525 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdfbbca-235e-4ebf-a5d3-f6de099df483 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:58.358394 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 205/418] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jul 15 19:14:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:14:58.359847 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8986d85c-1802-4c0d-9819-424f5b5738e0 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:58.360144 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8986d85c-1802-4c0d-9819-424f5b5738e0 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:58.360424 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8986d85c-1802-4c0d-9819-424f5b5738e0 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:58.361974 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 206/419] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:14:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:14:58.364562 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8986d85c-1802-4c0d-9819-424f5b5738e0 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:58.394912 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034c3110-f623-4731-87af-53aa8aefe5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:58.395721 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034c3110-f623-4731-87af-53aa8aefe5e3 None admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:58.395999 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034c3110-f623-4731-87af-53aa8aefe5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:58.396266 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034c3110-f623-4731-87af-53aa8aefe5e3 None admin] PATH_INFO: `/v3/projects/392b3b5584be451c8c287eb751b4d7de/groups/f0d021f243a44e049c331ebf4f87cbab/roles/4318f86ba9f74cd4a0bef1c311276657` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:58.398678 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034c3110-f623-4731-87af-53aa8aefe5e3 None admin] RBAC: Authorizing `identity:create_grant(project_id=392b3b5584be451c8c287eb751b4d7de, group_id=f0d021f243a44e049c331ebf4f87cbab, role_id=4318f86ba9f74cd4a0bef1c311276657)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:58.399950 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:58.400109 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034c3110-f623-4731-87af-53aa8aefe5e3 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:58.476736 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 214/420] 10.222.0.33 () {58 vars in 1442 bytes} [Fri Jul 15 19:14:58 2022] PUT /identity/v3/projects/392b3b5584be451c8c287eb751b4d7de/groups/f0d021f243a44e049c331ebf4f87cbab/roles/4318f86ba9f74cd4a0bef1c311276657 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:14:58.478430 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4265eba8-deb3-489a-8d14-e4308b98ea80 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:58.478725 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4265eba8-deb3-489a-8d14-e4308b98ea80 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:58.479005 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4265eba8-deb3-489a-8d14-e4308b98ea80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:58.483598 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8986d85c-1802-4c0d-9819-424f5b5738e0 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:58.514546 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-4265eba8-deb3-489a-8d14-e4308b98ea80 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:14:58.515791 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-4265eba8-deb3-489a-8d14-e4308b98ea80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:14:58.557379 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203b9089-693e-4d08-8533-aff2adc50793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:58.558086 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203b9089-693e-4d08-8533-aff2adc50793 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:58.558368 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203b9089-693e-4d08-8533-aff2adc50793 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:58.558646 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203b9089-693e-4d08-8533-aff2adc50793 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:58.669107 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:58.674639 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 215/421] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:14:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:14:58.688701 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203b9089-693e-4d08-8533-aff2adc50793 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:58.690223 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203b9089-693e-4d08-8533-aff2adc50793 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:58.704814 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4265eba8-deb3-489a-8d14-e4308b98ea80 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:58.732637 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:58.808649 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 207/422] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 15 19:14:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 15 19:14:58.814631 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203b9089-693e-4d08-8533-aff2adc50793 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:58.822585 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4265eba8-deb3-489a-8d14-e4308b98ea80 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:58.860057 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9f3f1b-090a-4d93-9baf-be577306c3e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:58.860968 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9f3f1b-090a-4d93-9baf-be577306c3e9 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:58.861249 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9f3f1b-090a-4d93-9baf-be577306c3e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:58.861522 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9f3f1b-090a-4d93-9baf-be577306c3e9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:58.873561 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9f3f1b-090a-4d93-9baf-be577306c3e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:14:58.881957 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-ad9f3f1b-090a-4d93-9baf-be577306c3e9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 19:14:58.881957 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:14:58.881957 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:14:58.881957 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:14:58.881957 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:14:58.881957 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:14:58.881957 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:14:58.881957 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:14:58.881957 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:14:58.881957 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:14:58.881957 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:14:58.881957 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:14:58.881957 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:14:58.881957 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:14:58.881957 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:14:58.881957 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:14:58.883002 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:14:58.883002 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:14:58.883002 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:14:58.883002 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:14:58.883002 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:14:58.883002 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:14:58.883002 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:14:58.883002 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:14:58.883002 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:14:58.883002 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:14:58.883002 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:14:58.883002 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:14:58.883002 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:14:58.883002 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:14:58.883002 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:14:58.883002 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:14:58.884199 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:14:58.884199 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:14:58.884199 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:14:58.884199 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:14:58.884199 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:14:58.884199 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:14:58.884199 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:14:58.884199 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:14:58.884199 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:14:58.884199 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:14:58.884199 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 19:14:58.884199 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:14:58.884199 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 216/423] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 15 19:14:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:14:58.889320 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9f3f1b-090a-4d93-9baf-be577306c3e9 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:58.993087 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203b9089-693e-4d08-8533-aff2adc50793 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:59.013142 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:59.057699 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9f3f1b-090a-4d93-9baf-be577306c3e9 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:59.072950 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:59.088574 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87372a34-18c9-428a-ac92-c54738b44a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:59.089496 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87372a34-18c9-428a-ac92-c54738b44a34 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:59.089787 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87372a34-18c9-428a-ac92-c54738b44a34 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:59.090135 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87372a34-18c9-428a-ac92-c54738b44a34 None admin] PATH_INFO: `/v3/projects/513641c037c84c54b779b5ca0684f17c` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:59.094605 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87372a34-18c9-428a-ac92-c54738b44a34 None admin] RBAC: Authorizing `identity:get_project(project_id=513641c037c84c54b779b5ca0684f17c)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:59.096074 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87372a34-18c9-428a-ac92-c54738b44a34 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:59.097282 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10b1f97-c3e8-4ce7-b404-b3b069e6d877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:59.097983 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10b1f97-c3e8-4ce7-b404-b3b069e6d877 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:59.098234 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 208/424] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 15 19:14:58 2022] GET /identity/v3/projects/513641c037c84c54b779b5ca0684f17c => generated 298 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:14:59.098371 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10b1f97-c3e8-4ce7-b404-b3b069e6d877 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:59.098550 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10b1f97-c3e8-4ce7-b404-b3b069e6d877 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:59.099277 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10b1f97-c3e8-4ce7-b404-b3b069e6d877 None admin] RBAC: Adding query filter params (name=member) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:59.099582 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10b1f97-c3e8-4ce7-b404-b3b069e6d877 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:59.100969 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10b1f97-c3e8-4ce7-b404-b3b069e6d877 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:59.103542 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87372a34-18c9-428a-ac92-c54738b44a34 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:59.111277 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 217/425] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jul 15 19:14:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:14:59.116979 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10b1f97-c3e8-4ce7-b404-b3b069e6d877 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:59.227865 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10b1f97-c3e8-4ce7-b404-b3b069e6d877 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:59.252323 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:59.265973 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02938395-f0a8-4f31-96d1-351758dadcfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:59.266630 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02938395-f0a8-4f31-96d1-351758dadcfa None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:59.266915 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02938395-f0a8-4f31-96d1-351758dadcfa None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:59.267298 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02938395-f0a8-4f31-96d1-351758dadcfa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:59.288086 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87372a34-18c9-428a-ac92-c54738b44a34 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:59.342792 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02938395-f0a8-4f31-96d1-351758dadcfa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:59.344246 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02938395-f0a8-4f31-96d1-351758dadcfa None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:59.364407 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1ef23a-c166-4a02-84ac-b065ab66f892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:59.365361 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1ef23a-c166-4a02-84ac-b065ab66f892 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:59.365737 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1ef23a-c166-4a02-84ac-b065ab66f892 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:59.366657 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1ef23a-c166-4a02-84ac-b065ab66f892 None admin] PATH_INFO: `/v3/groups/3d72e450754f4225bc7bc70c63a47fde` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:59.367925 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1ef23a-c166-4a02-84ac-b065ab66f892 None admin] RBAC: Authorizing `identity:get_group(group_id=3d72e450754f4225bc7bc70c63a47fde)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:59.369619 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1ef23a-c166-4a02-84ac-b065ab66f892 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:59.376557 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 209/426] 10.222.0.33 () {56 vars in 1180 bytes} [Fri Jul 15 19:14:59 2022] GET /identity/v3/groups/3d72e450754f4225bc7bc70c63a47fde => generated 226 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:14:59.378179 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b50837-face-48c0-b831-2214f4e167fd None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:59.378640 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b50837-face-48c0-b831-2214f4e167fd None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:59.378992 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b50837-face-48c0-b831-2214f4e167fd None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:59.380920 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 210/427] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:14:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:14:59.386705 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b50837-face-48c0-b831-2214f4e167fd None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:59.464918 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:59.486239 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 218/428] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 15 19:14:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 15 19:14:59.487788 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f1e12b-8c2f-4219-854d-b4407472bce3 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:59.488217 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f1e12b-8c2f-4219-854d-b4407472bce3 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:59.488725 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f1e12b-8c2f-4219-854d-b4407472bce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:59.515580 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-01f1e12b-8c2f-4219-854d-b4407472bce3 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:14:59.516120 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b50837-face-48c0-b831-2214f4e167fd None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:59.517928 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-01f1e12b-8c2f-4219-854d-b4407472bce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:14:59.543271 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:59.556487 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66200986-e23e-4e92-bb69-76912075f3ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:59.558156 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66200986-e23e-4e92-bb69-76912075f3ec None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:59.558156 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66200986-e23e-4e92-bb69-76912075f3ec None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:59.558156 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66200986-e23e-4e92-bb69-76912075f3ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:59.558664 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66200986-e23e-4e92-bb69-76912075f3ec None admin] RBAC: Adding query filter params (group.id=3d72e450754f4225bc7bc70c63a47fde, role.id=28763c6e70094116b3a4b2895895fa58, scope.project.id=513641c037c84c54b779b5ca0684f17c) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:59.558976 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66200986-e23e-4e92-bb69-76912075f3ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:59.560544 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66200986-e23e-4e92-bb69-76912075f3ec None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:59.574272 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 211/429] 10.222.0.33 () {56 vars in 1376 bytes} [Fri Jul 15 19:14:59 2022] GET /identity/v3/role_assignments?group.id=3d72e450754f4225bc7bc70c63a47fde&scope.project.id=513641c037c84c54b779b5ca0684f17c&role.id=28763c6e70094116b3a4b2895895fa58 => generated 611 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:59.575245 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66200986-e23e-4e92-bb69-76912075f3ec None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:59.607736 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 219/430] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:14:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:14:59.635741 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f1e12b-8c2f-4219-854d-b4407472bce3 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:59.761733 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f1e12b-8c2f-4219-854d-b4407472bce3 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:59.801995 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66200986-e23e-4e92-bb69-76912075f3ec None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:59.828249 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649fee71-c205-438c-81e9-53b779a22187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:59.829064 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649fee71-c205-438c-81e9-53b779a22187 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:59.829343 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649fee71-c205-438c-81e9-53b779a22187 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:59.829629 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649fee71-c205-438c-81e9-53b779a22187 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:59.846598 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649fee71-c205-438c-81e9-53b779a22187 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:14:59.853024 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:59.858839 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168c2d18-3191-4cc5-9825-93099a0c276b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:59.859718 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168c2d18-3191-4cc5-9825-93099a0c276b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:59.859998 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168c2d18-3191-4cc5-9825-93099a0c276b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:59.860270 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168c2d18-3191-4cc5-9825-93099a0c276b None admin] PATH_INFO: `/v3/projects/12e6f4bc187942989f83aeb1902a3036` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:59.861479 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168c2d18-3191-4cc5-9825-93099a0c276b None admin] RBAC: Authorizing `identity:get_project(project_id=12e6f4bc187942989f83aeb1902a3036)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:59.862556 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-649fee71-c205-438c-81e9-53b779a22187 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 19:14:59.862556 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:14:59.862556 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:14:59.862556 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:14:59.862556 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:14:59.862556 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:14:59.862556 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:14:59.862556 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:14:59.862556 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:14:59.862556 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:14:59.862556 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:14:59.862556 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:14:59.862556 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:14:59.862556 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:14:59.862556 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:14:59.862556 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:14:59.863461 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:14:59.863461 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:14:59.863461 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:14:59.863461 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:14:59.863461 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:14:59.863461 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:14:59.863461 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:14:59.863461 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:14:59.863461 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:14:59.863461 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:14:59.863461 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:14:59.863461 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:14:59.863461 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:14:59.863461 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:14:59.863461 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:14:59.863461 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:14:59.864280 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:14:59.864280 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:14:59.864280 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:14:59.864280 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:14:59.864280 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:14:59.864280 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:14:59.864280 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:14:59.864280 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:14:59.864280 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:14:59.864280 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:14:59.864280 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 19:14:59.864280 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:14:59.864967 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168c2d18-3191-4cc5-9825-93099a0c276b None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:59.864967 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 212/431] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 15 19:14:59 2022] GET /identity/v3/projects/12e6f4bc187942989f83aeb1902a3036 => generated 302 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:59.867940 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 220/432] 10.222.0.33 () {56 vars in 1114 bytes} [Fri Jul 15 19:14:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:14:59.871061 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168c2d18-3191-4cc5-9825-93099a0c276b None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:59.875940 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649fee71-c205-438c-81e9-53b779a22187 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:59.892832 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:00.018736 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649fee71-c205-438c-81e9-53b779a22187 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:00.023183 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168c2d18-3191-4cc5-9825-93099a0c276b None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:00.066755 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b84b3a-7550-4d5b-96b3-95674f982218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:00.067711 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b84b3a-7550-4d5b-96b3-95674f982218 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:00.068129 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b84b3a-7550-4d5b-96b3-95674f982218 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:00.068550 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b84b3a-7550-4d5b-96b3-95674f982218 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:00.069333 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c652da-aa44-4603-8375-a369ba4a6c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:00.069669 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b84b3a-7550-4d5b-96b3-95674f982218 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:00.070165 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b84b3a-7550-4d5b-96b3-95674f982218 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:00.070486 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c652da-aa44-4603-8375-a369ba4a6c11 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:00.070862 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c652da-aa44-4603-8375-a369ba4a6c11 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:00.071229 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c652da-aa44-4603-8375-a369ba4a6c11 None admin] PATH_INFO: `/v3/groups/3d72e450754f4225bc7bc70c63a47fde` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:00.072185 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b84b3a-7550-4d5b-96b3-95674f982218 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:00.072616 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c652da-aa44-4603-8375-a369ba4a6c11 None admin] RBAC: Authorizing `identity:get_group(group_id=3d72e450754f4225bc7bc70c63a47fde)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:00.074292 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c652da-aa44-4603-8375-a369ba4a6c11 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:00.076154 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 213/433] 10.222.0.33 () {56 vars in 1180 bytes} [Fri Jul 15 19:14:59 2022] GET /identity/v3/groups/3d72e450754f4225bc7bc70c63a47fde => generated 226 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:00.082055 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 221/434] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 15 19:14:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:15:00.082197 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c652da-aa44-4603-8375-a369ba4a6c11 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:00.089533 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b84b3a-7550-4d5b-96b3-95674f982218 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:00.176707 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:00.192430 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:00.234591 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c652da-aa44-4603-8375-a369ba4a6c11 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:00.237868 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b84b3a-7550-4d5b-96b3-95674f982218 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:00.277616 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eddb70c-9daa-4ceb-ac2d-d2ae37712fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:00.278293 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eddb70c-9daa-4ceb-ac2d-d2ae37712fb8 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:00.278570 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eddb70c-9daa-4ceb-ac2d-d2ae37712fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:00.278930 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eddb70c-9daa-4ceb-ac2d-d2ae37712fb8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:00.283179 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692ea36e-c5ea-43f2-b3de-7ba308ec2370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:00.284089 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ea36e-c5ea-43f2-b3de-7ba308ec2370 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:00.284735 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ea36e-c5ea-43f2-b3de-7ba308ec2370 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:00.285210 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ea36e-c5ea-43f2-b3de-7ba308ec2370 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:00.287179 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692ea36e-c5ea-43f2-b3de-7ba308ec2370 None admin] RBAC: Adding query filter params (group.id=3d72e450754f4225bc7bc70c63a47fde, role.id=28763c6e70094116b3a4b2895895fa58, scope.project.id=12e6f4bc187942989f83aeb1902a3036) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:00.287716 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692ea36e-c5ea-43f2-b3de-7ba308ec2370 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:00.289428 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692ea36e-c5ea-43f2-b3de-7ba308ec2370 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:00.297968 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 214/435] 10.222.0.33 () {56 vars in 1376 bytes} [Fri Jul 15 19:15:00 2022] GET /identity/v3/role_assignments?group.id=3d72e450754f4225bc7bc70c63a47fde&scope.project.id=12e6f4bc187942989f83aeb1902a3036&role.id=28763c6e70094116b3a4b2895895fa58 => generated 611 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:15:00.377955 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eddb70c-9daa-4ceb-ac2d-d2ae37712fb8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:00.379361 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eddb70c-9daa-4ceb-ac2d-d2ae37712fb8 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:00.425668 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb26b00-0d61-42bf-9b54-a683a84cd4ef None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:00.425996 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb26b00-0d61-42bf-9b54-a683a84cd4ef None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:00.426277 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb26b00-0d61-42bf-9b54-a683a84cd4ef None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:00.427677 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 215/436] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:15:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:15:00.434030 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b5725a-0466-49cf-bb05-6a0c06a37e5a None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:00.434315 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b5725a-0466-49cf-bb05-6a0c06a37e5a None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:00.434588 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b5725a-0466-49cf-bb05-6a0c06a37e5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:00.453326 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-92b5725a-0466-49cf-bb05-6a0c06a37e5a None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:15:00.454571 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-92b5725a-0466-49cf-bb05-6a0c06a37e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:15:00.479806 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:00.517667 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 222/437] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 15 19:15:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 15 19:15:00.522688 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eddb70c-9daa-4ceb-ac2d-d2ae37712fb8 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:00.554218 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 216/438] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:15:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 19:15:00.578533 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b5725a-0466-49cf-bb05-6a0c06a37e5a None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:00.644047 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eddb70c-9daa-4ceb-ac2d-d2ae37712fb8 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:00.665482 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:00.679000 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505875f1-5600-429e-b61b-e6ee7d140147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:00.679790 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505875f1-5600-429e-b61b-e6ee7d140147 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:00.680071 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505875f1-5600-429e-b61b-e6ee7d140147 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:00.680350 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505875f1-5600-429e-b61b-e6ee7d140147 None admin] PATH_INFO: `/v3/projects/513641c037c84c54b779b5ca0684f17c` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:00.681563 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505875f1-5600-429e-b61b-e6ee7d140147 None admin] RBAC: Authorizing `identity:get_project(project_id=513641c037c84c54b779b5ca0684f17c)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:00.683033 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505875f1-5600-429e-b61b-e6ee7d140147 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:00.684744 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 223/439] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 15 19:15:00 2022] GET /identity/v3/projects/513641c037c84c54b779b5ca0684f17c => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:00.690227 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505875f1-5600-429e-b61b-e6ee7d140147 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:00.707410 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b5725a-0466-49cf-bb05-6a0c06a37e5a None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:00.736255 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:00.744068 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f19447-2d90-4492-8a3d-f993455d338d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:00.744881 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f19447-2d90-4492-8a3d-f993455d338d None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:00.745162 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f19447-2d90-4492-8a3d-f993455d338d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:00.745441 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f19447-2d90-4492-8a3d-f993455d338d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:00.756066 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f19447-2d90-4492-8a3d-f993455d338d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:15:00.764903 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-62f19447-2d90-4492-8a3d-f993455d338d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 19:15:00.764903 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:00.764903 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:00.764903 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:00.764903 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:00.764903 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:00.764903 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:00.764903 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:00.764903 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:00.764903 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:00.764903 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:00.764903 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:00.764903 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:15:00.764903 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:15:00.764903 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:15:00.764903 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:15:00.765876 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:15:00.765876 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:15:00.765876 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:15:00.765876 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:15:00.765876 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:00.765876 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:00.765876 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:00.765876 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:00.765876 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:00.765876 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:00.765876 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:00.765876 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:00.765876 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:00.765876 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:00.765876 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:00.765876 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:00.766711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:00.766711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:00.766711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:00.766711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:00.766711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:15:00.766711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:15:00.766711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:15:00.766711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:15:00.766711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:15:00.766711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:15:00.766711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 19:15:00.766711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:00.766711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 217/440] 10.222.0.33 () {56 vars in 1114 bytes} [Fri Jul 15 19:15:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:15:00.771994 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f19447-2d90-4492-8a3d-f993455d338d None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:00.800105 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505875f1-5600-429e-b61b-e6ee7d140147 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:00.841796 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e2e5dc-6593-4512-bd97-095221233308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:00.842658 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e2e5dc-6593-4512-bd97-095221233308 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:00.842944 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e2e5dc-6593-4512-bd97-095221233308 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:00.843224 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e2e5dc-6593-4512-bd97-095221233308 None admin] PATH_INFO: `/v3/groups/3d72e450754f4225bc7bc70c63a47fde` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:00.844411 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e2e5dc-6593-4512-bd97-095221233308 None admin] RBAC: Authorizing `identity:get_group(group_id=3d72e450754f4225bc7bc70c63a47fde)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:00.845876 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e2e5dc-6593-4512-bd97-095221233308 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:00.847511 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 224/441] 10.222.0.33 () {56 vars in 1180 bytes} [Fri Jul 15 19:15:00 2022] GET /identity/v3/groups/3d72e450754f4225bc7bc70c63a47fde => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:15:00.853108 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e2e5dc-6593-4512-bd97-095221233308 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:00.892555 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f19447-2d90-4492-8a3d-f993455d338d None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:00.916956 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:00.927762 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307793b1-a742-4523-8b9d-4d441ff01afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:00.930804 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307793b1-a742-4523-8b9d-4d441ff01afe None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:00.930804 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307793b1-a742-4523-8b9d-4d441ff01afe None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:00.930804 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307793b1-a742-4523-8b9d-4d441ff01afe None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:00.930804 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307793b1-a742-4523-8b9d-4d441ff01afe None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:00.930804 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307793b1-a742-4523-8b9d-4d441ff01afe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:00.932024 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307793b1-a742-4523-8b9d-4d441ff01afe None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:00.946049 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 218/442] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 15 19:15:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:15:00.947395 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269f72c9-4b2b-42f2-ba61-067c99e3b97c None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:00.947694 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269f72c9-4b2b-42f2-ba61-067c99e3b97c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:00.947965 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269f72c9-4b2b-42f2-ba61-067c99e3b97c None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:00.949231 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 219/443] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:15:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:15:00.949975 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269f72c9-4b2b-42f2-ba61-067c99e3b97c None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:00.957334 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e2e5dc-6593-4512-bd97-095221233308 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:00.968038 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:00.992950 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12639703-beac-4c0c-9508-674cfc50299c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:00.993939 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12639703-beac-4c0c-9508-674cfc50299c None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:00.994224 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12639703-beac-4c0c-9508-674cfc50299c None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:00.994505 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12639703-beac-4c0c-9508-674cfc50299c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:00.995336 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12639703-beac-4c0c-9508-674cfc50299c None admin] RBAC: Adding query filter params (group.id=3d72e450754f4225bc7bc70c63a47fde, role.id=e97d26621445466d9550261c608db9c5, scope.project.id=513641c037c84c54b779b5ca0684f17c) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:00.995654 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12639703-beac-4c0c-9508-674cfc50299c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:00.997080 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12639703-beac-4c0c-9508-674cfc50299c None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:01.005521 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 225/444] 10.222.0.33 () {56 vars in 1376 bytes} [Fri Jul 15 19:15:00 2022] GET /identity/v3/role_assignments?group.id=3d72e450754f4225bc7bc70c63a47fde&scope.project.id=513641c037c84c54b779b5ca0684f17c&role.id=e97d26621445466d9550261c608db9c5 => generated 611 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:15:01.007396 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473d3c04-fd32-46a8-9c84-18311ac727cf None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:01.007598 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473d3c04-fd32-46a8-9c84-18311ac727cf None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:01.007870 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473d3c04-fd32-46a8-9c84-18311ac727cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:01.035347 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-473d3c04-fd32-46a8-9c84-18311ac727cf None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:15:01.036643 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-473d3c04-fd32-46a8-9c84-18311ac727cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:15:01.064775 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269f72c9-4b2b-42f2-ba61-067c99e3b97c None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:01.104017 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1176f0f6-d573-488c-b2d3-57e83b683fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:01.104689 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1176f0f6-d573-488c-b2d3-57e83b683fae None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:01.104969 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1176f0f6-d573-488c-b2d3-57e83b683fae None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:01.105243 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1176f0f6-d573-488c-b2d3-57e83b683fae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:01.114147 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 226/445] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:15:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:15:01.139316 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473d3c04-fd32-46a8-9c84-18311ac727cf None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:01.169344 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:01.189681 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1176f0f6-d573-488c-b2d3-57e83b683fae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:01.191474 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1176f0f6-d573-488c-b2d3-57e83b683fae None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:01.260448 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473d3c04-fd32-46a8-9c84-18311ac727cf None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:01.263350 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:01.302029 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3641675a-33d1-4712-9b85-727bcda49c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:01.302837 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3641675a-33d1-4712-9b85-727bcda49c7a None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:01.303120 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3641675a-33d1-4712-9b85-727bcda49c7a None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:01.303994 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3641675a-33d1-4712-9b85-727bcda49c7a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:01.318100 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 220/446] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 15 19:15:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 19:15:01.320429 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3641675a-33d1-4712-9b85-727bcda49c7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:15:01.324552 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1176f0f6-d573-488c-b2d3-57e83b683fae None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:01.330999 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-3641675a-33d1-4712-9b85-727bcda49c7a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:15:01.330999 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:01.330999 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:01.330999 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:01.330999 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:01.330999 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:01.330999 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:01.330999 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:01.330999 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:01.330999 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:01.330999 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:01.330999 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:01.330999 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:15:01.330999 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:15:01.330999 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:15:01.330999 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:15:01.331971 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:15:01.331971 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:15:01.331971 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:15:01.331971 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:15:01.331971 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:01.331971 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:01.331971 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:01.331971 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:01.331971 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:01.331971 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:01.331971 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:01.331971 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:01.331971 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:01.331971 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:01.331971 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:01.331971 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:01.332778 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:01.332778 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:01.332778 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:01.332778 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:01.332778 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:15:01.332778 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:15:01.332778 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:15:01.332778 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:15:01.332778 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:15:01.332778 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:15:01.332778 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:15:01.332778 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:01.332778 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 227/447] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jul 15 19:15:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:15:01.340140 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3641675a-33d1-4712-9b85-727bcda49c7a None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:01.425040 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:01.442449 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1176f0f6-d573-488c-b2d3-57e83b683fae None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:01.471720 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3641675a-33d1-4712-9b85-727bcda49c7a None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:01.492870 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70afdf7d-5985-46c4-b26d-28aed1e73697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:01.494440 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70afdf7d-5985-46c4-b26d-28aed1e73697 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:01.495222 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70afdf7d-5985-46c4-b26d-28aed1e73697 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:01.495525 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70afdf7d-5985-46c4-b26d-28aed1e73697 None admin] PATH_INFO: `/v3/projects/12e6f4bc187942989f83aeb1902a3036` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:01.497658 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70afdf7d-5985-46c4-b26d-28aed1e73697 None admin] RBAC: Authorizing `identity:get_project(project_id=12e6f4bc187942989f83aeb1902a3036)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:01.500005 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70afdf7d-5985-46c4-b26d-28aed1e73697 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:01.501818 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 221/448] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 15 19:15:01 2022] GET /identity/v3/projects/12e6f4bc187942989f83aeb1902a3036 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:15:01.507420 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70afdf7d-5985-46c4-b26d-28aed1e73697 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:01.523012 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc8f4ef-3f0e-4099-ab3c-49991e454f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:01.523687 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc8f4ef-3f0e-4099-ab3c-49991e454f93 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:01.523976 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc8f4ef-3f0e-4099-ab3c-49991e454f93 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:01.524257 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc8f4ef-3f0e-4099-ab3c-49991e454f93 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:01.525006 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc8f4ef-3f0e-4099-ab3c-49991e454f93 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:01.525310 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc8f4ef-3f0e-4099-ab3c-49991e454f93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:01.526719 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc8f4ef-3f0e-4099-ab3c-49991e454f93 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:01.534950 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 228/449] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 15 19:15:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:15:01.541479 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc8f4ef-3f0e-4099-ab3c-49991e454f93 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:01.569247 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:01.611083 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70afdf7d-5985-46c4-b26d-28aed1e73697 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:01.642221 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc8f4ef-3f0e-4099-ab3c-49991e454f93 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:01.644420 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a45261e-0ec6-4763-82bf-9bf677add747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:01.645284 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a45261e-0ec6-4763-82bf-9bf677add747 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:01.645566 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a45261e-0ec6-4763-82bf-9bf677add747 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:01.645875 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a45261e-0ec6-4763-82bf-9bf677add747 None admin] PATH_INFO: `/v3/groups/3d72e450754f4225bc7bc70c63a47fde` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:01.647572 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a45261e-0ec6-4763-82bf-9bf677add747 None admin] RBAC: Authorizing `identity:get_group(group_id=3d72e450754f4225bc7bc70c63a47fde)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:01.649296 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a45261e-0ec6-4763-82bf-9bf677add747 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:01.650967 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 222/450] 10.222.0.33 () {56 vars in 1180 bytes} [Fri Jul 15 19:15:01 2022] GET /identity/v3/groups/3d72e450754f4225bc7bc70c63a47fde => generated 226 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:01.656832 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a45261e-0ec6-4763-82bf-9bf677add747 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:01.676850 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60057399-4457-41a2-83cb-b2583dd2cc10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:01.677554 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60057399-4457-41a2-83cb-b2583dd2cc10 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:01.677907 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60057399-4457-41a2-83cb-b2583dd2cc10 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:01.678197 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60057399-4457-41a2-83cb-b2583dd2cc10 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:01.678713 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:01.745786 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60057399-4457-41a2-83cb-b2583dd2cc10 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:01.747271 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60057399-4457-41a2-83cb-b2583dd2cc10 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:01.756868 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a45261e-0ec6-4763-82bf-9bf677add747 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:01.772615 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:01.793309 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acd2b5e-76f1-4981-b040-3a2b10e7d6b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:01.794054 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd2b5e-76f1-4981-b040-3a2b10e7d6b7 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:01.794349 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd2b5e-76f1-4981-b040-3a2b10e7d6b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:01.794622 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd2b5e-76f1-4981-b040-3a2b10e7d6b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:01.795457 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acd2b5e-76f1-4981-b040-3a2b10e7d6b7 None admin] RBAC: Adding query filter params (group.id=3d72e450754f4225bc7bc70c63a47fde, role.id=e97d26621445466d9550261c608db9c5, scope.project.id=12e6f4bc187942989f83aeb1902a3036) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:01.795768 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acd2b5e-76f1-4981-b040-3a2b10e7d6b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:01.797217 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acd2b5e-76f1-4981-b040-3a2b10e7d6b7 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:01.810147 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 223/451] 10.222.0.33 () {56 vars in 1376 bytes} [Fri Jul 15 19:15:01 2022] GET /identity/v3/role_assignments?group.id=3d72e450754f4225bc7bc70c63a47fde&scope.project.id=12e6f4bc187942989f83aeb1902a3036&role.id=e97d26621445466d9550261c608db9c5 => generated 611 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:15:01.849478 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 229/452] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 15 19:15:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 15 19:15:01.856108 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acd2b5e-76f1-4981-b040-3a2b10e7d6b7 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:01.945715 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:01.961817 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acd2b5e-76f1-4981-b040-3a2b10e7d6b7 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:01.997162 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74de3a35-1f18-4bbc-bbe3-13ba9678c56f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:01.998061 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74de3a35-1f18-4bbc-bbe3-13ba9678c56f None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:01.998345 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74de3a35-1f18-4bbc-bbe3-13ba9678c56f None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:01.998624 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74de3a35-1f18-4bbc-bbe3-13ba9678c56f None admin] PATH_INFO: `/v3/projects/392b3b5584be451c8c287eb751b4d7de` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:01.999853 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74de3a35-1f18-4bbc-bbe3-13ba9678c56f None admin] RBAC: Authorizing `identity:get_project(project_id=392b3b5584be451c8c287eb751b4d7de)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:02.001289 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74de3a35-1f18-4bbc-bbe3-13ba9678c56f None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:02.003042 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 224/453] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 15 19:15:01 2022] GET /identity/v3/projects/392b3b5584be451c8c287eb751b4d7de => generated 344 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:02.009161 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60057399-4457-41a2-83cb-b2583dd2cc10 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:02.111101 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60057399-4457-41a2-83cb-b2583dd2cc10 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:02.136606 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:02.145359 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f70c52-de56-4949-b81d-06b7df560f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:02.146270 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f70c52-de56-4949-b81d-06b7df560f54 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:02.146552 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f70c52-de56-4949-b81d-06b7df560f54 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:02.146831 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f70c52-de56-4949-b81d-06b7df560f54 None admin] PATH_INFO: `/v3/groups/f0d021f243a44e049c331ebf4f87cbab` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:02.148802 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f70c52-de56-4949-b81d-06b7df560f54 None admin] RBAC: Authorizing `identity:get_group(group_id=f0d021f243a44e049c331ebf4f87cbab)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:02.150332 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f70c52-de56-4949-b81d-06b7df560f54 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:02.151961 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 230/454] 10.222.0.33 () {56 vars in 1180 bytes} [Fri Jul 15 19:15:02 2022] GET /identity/v3/groups/f0d021f243a44e049c331ebf4f87cbab => generated 229 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:02.157949 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74de3a35-1f18-4bbc-bbe3-13ba9678c56f None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:02.254604 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74de3a35-1f18-4bbc-bbe3-13ba9678c56f None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:02.287508 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e022c33c-d157-4b66-ae6a-2a1715e03966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:02.288169 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e022c33c-d157-4b66-ae6a-2a1715e03966 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:02.288452 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e022c33c-d157-4b66-ae6a-2a1715e03966 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:02.288729 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e022c33c-d157-4b66-ae6a-2a1715e03966 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:02.289555 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e022c33c-d157-4b66-ae6a-2a1715e03966 None admin] RBAC: Adding query filter params (group.id=f0d021f243a44e049c331ebf4f87cbab, role.id=4318f86ba9f74cd4a0bef1c311276657, scope.project.id=392b3b5584be451c8c287eb751b4d7de) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:02.289894 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e022c33c-d157-4b66-ae6a-2a1715e03966 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:02.291330 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e022c33c-d157-4b66-ae6a-2a1715e03966 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:02.297002 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 225/455] 10.222.0.33 () {56 vars in 1376 bytes} [Fri Jul 15 19:15:02 2022] GET /identity/v3/role_assignments?group.id=f0d021f243a44e049c331ebf4f87cbab&scope.project.id=392b3b5584be451c8c287eb751b4d7de&role.id=4318f86ba9f74cd4a0bef1c311276657 => generated 611 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:04.900051 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a45146e-5a5f-4b48-9aed-7044eb6923fc None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:04.900586 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a45146e-5a5f-4b48-9aed-7044eb6923fc None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:04.900586 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a45146e-5a5f-4b48-9aed-7044eb6923fc None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:04.902273 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 231/456] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:15:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:15:04.907596 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cffa61-c380-443e-9e79-56e7e64fe4e0 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:04.907976 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cffa61-c380-443e-9e79-56e7e64fe4e0 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:04.908162 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cffa61-c380-443e-9e79-56e7e64fe4e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:04.934893 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-c4cffa61-c380-443e-9e79-56e7e64fe4e0 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:15:04.936098 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-c4cffa61-c380-443e-9e79-56e7e64fe4e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:15:04.951355 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:05.003068 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd109a11-0344-4d29-a74a-2cd793f878fc None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:05.003403 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd109a11-0344-4d29-a74a-2cd793f878fc None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:05.003629 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd109a11-0344-4d29-a74a-2cd793f878fc None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:05.005275 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 232/457] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:15:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:15:05.012211 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76361806-0465-4c2f-8abb-17efa1bf3e97 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:05.012748 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76361806-0465-4c2f-8abb-17efa1bf3e97 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:05.013228 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76361806-0465-4c2f-8abb-17efa1bf3e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:05.037956 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 226/458] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:15:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:15:05.046783 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-76361806-0465-4c2f-8abb-17efa1bf3e97 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:15:05.048225 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-76361806-0465-4c2f-8abb-17efa1bf3e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:15:05.054317 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cffa61-c380-443e-9e79-56e7e64fe4e0 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:05.150032 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 233/459] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:15:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:15:05.164112 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6bf427-e20b-44ba-a201-7165b26dce73 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:05.164418 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6bf427-e20b-44ba-a201-7165b26dce73 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:05.164671 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6bf427-e20b-44ba-a201-7165b26dce73 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:05.166219 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 234/460] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:15:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 19:15:05.173572 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03200717-78ef-43eb-8447-b28a42cff9ad None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:05.173893 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03200717-78ef-43eb-8447-b28a42cff9ad None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:05.174178 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03200717-78ef-43eb-8447-b28a42cff9ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:05.193600 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-03200717-78ef-43eb-8447-b28a42cff9ad None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:15:05.195079 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-03200717-78ef-43eb-8447-b28a42cff9ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:15:05.227395 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cffa61-c380-443e-9e79-56e7e64fe4e0 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:05.271197 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d76cde-399d-4f1d-bf89-41ed8c8c95e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:05.272902 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d76cde-399d-4f1d-bf89-41ed8c8c95e4 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:05.273388 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d76cde-399d-4f1d-bf89-41ed8c8c95e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:05.273966 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d76cde-399d-4f1d-bf89-41ed8c8c95e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:05.279332 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 235/461] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:15:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 19:15:05.280196 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03200717-78ef-43eb-8447-b28a42cff9ad None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:05.286936 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d76cde-399d-4f1d-bf89-41ed8c8c95e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:05.288945 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d76cde-399d-4f1d-bf89-41ed8c8c95e4 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:05.298360 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:05.313131 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-35d76cde-399d-4f1d-bf89-41ed8c8c95e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:05.313131 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:05.313131 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:05.313131 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:05.313131 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:05.313131 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:05.313131 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:05.313131 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:05.313131 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:05.313131 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:05.313131 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:05.313131 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:05.313131 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:15:05.313131 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:15:05.313131 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:15:05.314098 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:15:05.314098 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:05.314098 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:05.314098 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:05.314098 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:05.314098 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:05.314098 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:05.314098 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:05.314098 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:05.314098 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:05.314098 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:05.314098 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:05.314098 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:05.314098 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:05.314098 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:05.314098 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:05.314098 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:05.314942 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:15:05.314942 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:15:05.314942 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:05.314942 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:05.314942 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 227/462] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:15:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:05.318917 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d76cde-399d-4f1d-bf89-41ed8c8c95e4 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:05.344227 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:05.391410 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03200717-78ef-43eb-8447-b28a42cff9ad None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:05.439353 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948859e6-114b-476e-944b-e8eb3066fb96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:05.440069 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948859e6-114b-476e-944b-e8eb3066fb96 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:05.440349 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948859e6-114b-476e-944b-e8eb3066fb96 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:05.440623 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948859e6-114b-476e-944b-e8eb3066fb96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:05.443188 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d76cde-399d-4f1d-bf89-41ed8c8c95e4 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:05.451393 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948859e6-114b-476e-944b-e8eb3066fb96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:05.452829 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948859e6-114b-476e-944b-e8eb3066fb96 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:05.470887 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-948859e6-114b-476e-944b-e8eb3066fb96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:05.470887 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:05.470887 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:05.470887 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:05.470887 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:05.470887 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:05.470887 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:05.470887 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:05.470887 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:05.470887 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:05.470887 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:05.470887 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:05.470887 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:15:05.470887 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:15:05.470887 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:15:05.471742 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:15:05.471742 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:05.471742 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:05.471742 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:05.471742 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:05.471742 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:05.471742 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:05.471742 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:05.471742 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:05.471742 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:05.471742 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:05.471742 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:05.471742 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:05.471742 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:05.471742 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:05.471742 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:05.471742 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:05.472566 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:15:05.472566 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:15:05.472566 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:05.472566 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:05.472566 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 236/463] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:15:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:05.472832 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948859e6-114b-476e-944b-e8eb3066fb96 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:05.485058 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e32a5e-433a-422f-94a1-0c33b4450e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:05.485895 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e32a5e-433a-422f-94a1-0c33b4450e83 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:05.486157 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e32a5e-433a-422f-94a1-0c33b4450e83 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:05.486431 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e32a5e-433a-422f-94a1-0c33b4450e83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:05.498691 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e32a5e-433a-422f-94a1-0c33b4450e83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:05.500155 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e32a5e-433a-422f-94a1-0c33b4450e83 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:05.522010 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-17e32a5e-433a-422f-94a1-0c33b4450e83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:05.522010 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:05.522010 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:05.522010 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:05.522010 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:05.522010 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:05.522010 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:05.522010 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:05.522010 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:05.522010 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:05.522010 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:05.522010 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:05.522010 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:15:05.522010 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:15:05.522010 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:15:05.523269 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:15:05.523269 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:05.523269 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:05.523269 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:05.523269 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:05.523269 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:05.523269 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:05.523269 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:05.523269 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:05.523269 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:05.523269 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:05.523269 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:05.523269 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:05.523269 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:05.523269 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:05.523269 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:05.523269 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:05.524223 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:15:05.524223 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:15:05.524223 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:05.524223 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:05.531052 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 228/464] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:15:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:05.531052 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df690c44-bfdb-494a-9572-7403c63fbd87 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:05.531052 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df690c44-bfdb-494a-9572-7403c63fbd87 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:05.531052 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df690c44-bfdb-494a-9572-7403c63fbd87 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:05.531052 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 229/465] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:15:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:15:05.531656 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df690c44-bfdb-494a-9572-7403c63fbd87 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:05.592987 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:05.594290 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948859e6-114b-476e-944b-e8eb3066fb96 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:05.602805 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:05.631237 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e98ae5-8e76-4e02-af73-7868078a6657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:05.632605 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e98ae5-8e76-4e02-af73-7868078a6657 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:05.632907 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e98ae5-8e76-4e02-af73-7868078a6657 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:05.633203 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e98ae5-8e76-4e02-af73-7868078a6657 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:05.633968 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e98ae5-8e76-4e02-af73-7868078a6657 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:05.634300 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e98ae5-8e76-4e02-af73-7868078a6657 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:05.635677 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e98ae5-8e76-4e02-af73-7868078a6657 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:05.647247 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 237/466] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:15:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:05.648655 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e98ae5-8e76-4e02-af73-7868078a6657 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:05.653673 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df690c44-bfdb-494a-9572-7403c63fbd87 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:05.690548 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964eaa37-23f7-4a36-9a49-5a971d74e907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:05.691136 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964eaa37-23f7-4a36-9a49-5a971d74e907 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:05.691418 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964eaa37-23f7-4a36-9a49-5a971d74e907 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:05.691690 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964eaa37-23f7-4a36-9a49-5a971d74e907 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:05.692421 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964eaa37-23f7-4a36-9a49-5a971d74e907 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:05.692725 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964eaa37-23f7-4a36-9a49-5a971d74e907 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:05.694984 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964eaa37-23f7-4a36-9a49-5a971d74e907 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:05.707142 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 230/467] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:15:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:05.708219 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c5a00-b34c-4721-8513-54cb241497a7 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:05.708510 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c5a00-b34c-4721-8513-54cb241497a7 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:05.708789 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c5a00-b34c-4721-8513-54cb241497a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:05.725894 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-7a2c5a00-b34c-4721-8513-54cb241497a7 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:15:05.727104 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-7a2c5a00-b34c-4721-8513-54cb241497a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:15:05.762049 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e98ae5-8e76-4e02-af73-7868078a6657 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:05.798479 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 231/468] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:15:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:15:05.799133 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2c5a00-b34c-4721-8513-54cb241497a7 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:05.809741 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8377e6-0247-47cc-921f-3aa252bc1df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:05.810449 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8377e6-0247-47cc-921f-3aa252bc1df8 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:05.810725 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8377e6-0247-47cc-921f-3aa252bc1df8 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:05.810995 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8377e6-0247-47cc-921f-3aa252bc1df8 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:05.811718 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8377e6-0247-47cc-921f-3aa252bc1df8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:05.812022 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8377e6-0247-47cc-921f-3aa252bc1df8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:05.813371 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8377e6-0247-47cc-921f-3aa252bc1df8 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:05.832501 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 238/469] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:15:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:05.832501 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8377e6-0247-47cc-921f-3aa252bc1df8 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:05.857975 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:05.861120 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:05.925440 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2c5a00-b34c-4721-8513-54cb241497a7 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:05.941699 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8377e6-0247-47cc-921f-3aa252bc1df8 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:05.962773 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c51ec3-1ecd-4987-823b-c498466c00e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:05.963441 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c51ec3-1ecd-4987-823b-c498466c00e8 None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:05.963717 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c51ec3-1ecd-4987-823b-c498466c00e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:05.963987 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c51ec3-1ecd-4987-823b-c498466c00e8 None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:05.964685 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c51ec3-1ecd-4987-823b-c498466c00e8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:05.966142 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c51ec3-1ecd-4987-823b-c498466c00e8 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:05.980652 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec082e50-d26b-4ad8-83c1-bbcf63f4694f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:05.981319 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec082e50-d26b-4ad8-83c1-bbcf63f4694f None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:05.981598 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec082e50-d26b-4ad8-83c1-bbcf63f4694f None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:05.981904 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec082e50-d26b-4ad8-83c1-bbcf63f4694f None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:05.982601 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec082e50-d26b-4ad8-83c1-bbcf63f4694f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:05.984034 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec082e50-d26b-4ad8-83c1-bbcf63f4694f None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:06.093789 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 232/470] 10.222.0.33 () {60 vars in 1132 bytes} [Fri Jul 15 19:15:05 2022] POST /identity/v3/users => generated 246 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:15:06.093789 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c51ec3-1ecd-4987-823b-c498466c00e8 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:06.141244 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 239/471] 10.222.0.33 () {60 vars in 1132 bytes} [Fri Jul 15 19:15:05 2022] POST /identity/v3/users => generated 248 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:15:06.142124 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec082e50-d26b-4ad8-83c1-bbcf63f4694f None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:06.215636 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:06.225188 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c51ec3-1ecd-4987-823b-c498466c00e8 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:06.243582 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:06.268120 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec082e50-d26b-4ad8-83c1-bbcf63f4694f None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:06.283637 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a24e5e-87e9-486f-890c-4e9b18f33fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:06.286056 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a24e5e-87e9-486f-890c-4e9b18f33fd8 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:06.286834 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a24e5e-87e9-486f-890c-4e9b18f33fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:06.287562 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a24e5e-87e9-486f-890c-4e9b18f33fd8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:06.302822 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a24e5e-87e9-486f-890c-4e9b18f33fd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:06.304896 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a24e5e-87e9-486f-890c-4e9b18f33fd8 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:06.312377 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ff9855-deeb-4cac-ac33-f001504a6c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:06.313062 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ff9855-deeb-4cac-ac33-f001504a6c73 None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:06.317936 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ff9855-deeb-4cac-ac33-f001504a6c73 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:06.317936 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ff9855-deeb-4cac-ac33-f001504a6c73 None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:06.317936 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ff9855-deeb-4cac-ac33-f001504a6c73 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:06.317936 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ff9855-deeb-4cac-ac33-f001504a6c73 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:06.327656 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-15a24e5e-87e9-486f-890c-4e9b18f33fd8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:06.327656 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:06.327656 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:06.327656 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:06.327656 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:06.327656 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:06.327656 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:06.327656 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:06.327656 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:06.327656 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:06.327656 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:06.327656 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:06.327656 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:15:06.327656 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:15:06.327656 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:15:06.328769 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:15:06.328769 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:06.328769 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:06.328769 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:06.328769 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:06.328769 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:06.328769 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:06.328769 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:06.328769 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:06.328769 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:06.328769 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:06.328769 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:06.328769 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:06.328769 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:06.328769 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:06.328769 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:06.328769 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:06.329644 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:15:06.329644 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:15:06.329644 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:06.329644 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:06.329644 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 233/472] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:15:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:06.330512 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee11f06-0059-4ced-b90a-d84af64d40ec None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:06.330796 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee11f06-0059-4ced-b90a-d84af64d40ec None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:06.331069 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee11f06-0059-4ced-b90a-d84af64d40ec None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:06.336860 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 234/473] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:15:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:15:06.336860 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee11f06-0059-4ced-b90a-d84af64d40ec None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:06.468218 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee11f06-0059-4ced-b90a-d84af64d40ec None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:06.473449 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 240/474] 10.222.0.33 () {60 vars in 1132 bytes} [Fri Jul 15 19:15:06 2022] POST /identity/v3/users => generated 248 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:15:06.474597 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c0077-a31b-4d93-8756-3b457c6a0fae None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:06.474887 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c0077-a31b-4d93-8756-3b457c6a0fae None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:06.475733 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c0077-a31b-4d93-8756-3b457c6a0fae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:06.495068 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-6a8c0077-a31b-4d93-8756-3b457c6a0fae None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:15:06.496748 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-6a8c0077-a31b-4d93-8756-3b457c6a0fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:15:06.513929 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f6cd71-e676-4a3f-8746-647ed27ba20a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:06.514990 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f6cd71-e676-4a3f-8746-647ed27ba20a None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:06.515496 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f6cd71-e676-4a3f-8746-647ed27ba20a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:06.516022 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f6cd71-e676-4a3f-8746-647ed27ba20a None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:06.517142 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f6cd71-e676-4a3f-8746-647ed27ba20a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:06.517702 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f6cd71-e676-4a3f-8746-647ed27ba20a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:06.519458 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f6cd71-e676-4a3f-8746-647ed27ba20a None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:06.535350 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 235/475] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:15:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:06.541779 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f6cd71-e676-4a3f-8746-647ed27ba20a None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:06.554977 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:06.633116 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 241/476] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:15:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:15:06.666214 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f6cd71-e676-4a3f-8746-647ed27ba20a None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:06.669050 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8c0077-a31b-4d93-8756-3b457c6a0fae None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:06.706476 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f900f526-b166-477d-9f92-2b7d8b317aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:06.707489 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f900f526-b166-477d-9f92-2b7d8b317aa2 None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:06.708000 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f900f526-b166-477d-9f92-2b7d8b317aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:06.708477 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f900f526-b166-477d-9f92-2b7d8b317aa2 None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:06.710107 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f900f526-b166-477d-9f92-2b7d8b317aa2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:06.711869 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f900f526-b166-477d-9f92-2b7d8b317aa2 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:06.767132 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:06.807001 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 236/477] 10.222.0.33 () {60 vars in 1132 bytes} [Fri Jul 15 19:15:06 2022] POST /identity/v3/users => generated 249 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:15:06.812364 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8c0077-a31b-4d93-8756-3b457c6a0fae None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:06.853964 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412369f0-2678-4266-9b52-b0cc5b71ef23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:06.854646 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412369f0-2678-4266-9b52-b0cc5b71ef23 None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:06.854925 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412369f0-2678-4266-9b52-b0cc5b71ef23 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:06.855197 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412369f0-2678-4266-9b52-b0cc5b71ef23 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:06.856073 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412369f0-2678-4266-9b52-b0cc5b71ef23 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:06.857545 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412369f0-2678-4266-9b52-b0cc5b71ef23 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:06.912246 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.sql.core [None req-412369f0-2678-4266-9b52-b0cc5b71ef23 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 15 19:15:06.912246 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 15 19:15:06.912246 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [parameters: {'id': '50bfb4223594435ab9933d6a0bba9489', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 15 19:15:06.912246 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80268) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 15 19:15:06.938186 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-412369f0-2678-4266-9b52-b0cc5b71ef23 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 15 19:15:06.938186 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:06.938186 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 15 19:15:06.938186 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 15 19:15:06.938186 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 15 19:15:06.938186 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 15 19:15:06.938186 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 15 19:15:06.938186 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application result = self._query(query) Jul 15 19:15:06.938186 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 15 19:15:06.938186 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application conn.query(q) Jul 15 19:15:06.938186 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 15 19:15:06.938186 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 15 19:15:06.938186 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 15 19:15:06.938186 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application result.read() Jul 15 19:15:06.938186 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 15 19:15:06.939159 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 15 19:15:06.939159 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 15 19:15:06.939159 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 15 19:15:06.939159 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 15 19:15:06.939159 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 15 19:15:06.939159 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 15 19:15:06.939159 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 15 19:15:06.939159 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 15 19:15:06.939159 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:06.939159 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 15 19:15:06.939159 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:06.939159 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:06.939159 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 15 19:15:06.939159 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 15 19:15:06.939159 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 15 19:15:06.939159 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return ref.to_dict() Jul 15 19:15:06.939159 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 15 19:15:06.939159 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application next(self.gen) Jul 15 19:15:06.940103 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 15 19:15:06.940103 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application yield resource Jul 15 19:15:06.940103 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 15 19:15:06.940103 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application next(self.gen) Jul 15 19:15:06.940103 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 15 19:15:06.940103 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self.session.rollback() Jul 15 19:15:06.940103 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 15 19:15:06.940103 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self.force_reraise() Jul 15 19:15:06.940103 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 15 19:15:06.940103 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise self.value Jul 15 19:15:06.940103 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 15 19:15:06.940103 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 15 19:15:06.940103 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 15 19:15:06.940103 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application session.commit() Jul 15 19:15:06.940103 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 15 19:15:06.940103 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 15 19:15:06.940103 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 15 19:15:06.940979 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._prepare_impl() Jul 15 19:15:06.940979 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 15 19:15:06.940979 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self.session.flush() Jul 15 19:15:06.940979 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 15 19:15:06.940979 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._flush(objects) Jul 15 19:15:06.940979 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 15 19:15:06.940979 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 15 19:15:06.940979 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 15 19:15:06.940979 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application compat.raise_( Jul 15 19:15:06.940979 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 15 19:15:06.940979 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception Jul 15 19:15:06.940979 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 15 19:15:06.940979 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application flush_context.execute() Jul 15 19:15:06.940979 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 15 19:15:06.940979 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rec.execute(self) Jul 15 19:15:06.940979 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 15 19:15:06.940979 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 15 19:15:06.943195 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 15 19:15:06.943195 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 15 19:15:06.943195 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 15 19:15:06.943195 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 15 19:15:06.943195 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 15 19:15:06.943195 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 15 19:15:06.943195 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 15 19:15:06.943195 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 15 19:15:06.943195 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 15 19:15:06.943195 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 15 19:15:06.943195 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 15 19:15:06.943195 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 15 19:15:06.943195 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 15 19:15:06.943195 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 15 19:15:06.943195 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 15 19:15:06.943195 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception Jul 15 19:15:06.948397 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 15 19:15:06.948397 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 15 19:15:06.948397 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 15 19:15:06.948397 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 15 19:15:06.948397 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 15 19:15:06.948397 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application result = self._query(query) Jul 15 19:15:06.948397 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 15 19:15:06.948397 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application conn.query(q) Jul 15 19:15:06.948397 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 15 19:15:06.948397 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 15 19:15:06.948397 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 15 19:15:06.948397 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application result.read() Jul 15 19:15:06.948397 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 15 19:15:06.948397 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 15 19:15:06.948397 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 15 19:15:06.948397 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 15 19:15:06.950407 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 15 19:15:06.950407 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 15 19:15:06.950407 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 15 19:15:06.950407 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 15 19:15:06.950407 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 15 19:15:06.950407 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 15 19:15:06.950407 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [parameters: {'id': '50bfb4223594435ab9933d6a0bba9489', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 15 19:15:06.950407 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 15 19:15:06.950407 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:06.950407 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 19:15:06.950407 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:06.950407 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:06.950407 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:06.950407 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:06.950407 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:06.950407 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:06.952567 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:06.952567 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:06.952567 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:06.952567 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:06.952567 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:06.952567 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:06.952567 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 15 19:15:06.952567 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 15 19:15:06.952567 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:06.952567 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:06.952567 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 15 19:15:06.952567 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 15 19:15:06.952567 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 15 19:15:06.952567 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 15 19:15:06.952567 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 15 19:15:06.952567 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:06.954664 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 242/478] 10.222.0.33 () {60 vars in 1131 bytes} [Fri Jul 15 19:15:06 2022] POST /identity/v3/roles => generated 144 bytes in 271 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:06.954810 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f900f526-b166-477d-9f92-2b7d8b317aa2 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:07.033373 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:07.077295 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f900f526-b166-477d-9f92-2b7d8b317aa2 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:07.113050 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e2c7a7-5b66-4653-86e7-a2aa77c3f2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:07.114270 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e2c7a7-5b66-4653-86e7-a2aa77c3f2b9 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:07.114759 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e2c7a7-5b66-4653-86e7-a2aa77c3f2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:07.115255 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e2c7a7-5b66-4653-86e7-a2aa77c3f2b9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:07.127398 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e2c7a7-5b66-4653-86e7-a2aa77c3f2b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:15:07.136560 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-98e2c7a7-5b66-4653-86e7-a2aa77c3f2b9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 19:15:07.136560 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:07.136560 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:07.136560 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:07.136560 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:07.136560 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:07.136560 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:07.136560 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:07.136560 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:07.136560 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:07.136560 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:07.136560 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:07.136560 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:15:07.136560 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:15:07.136560 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:15:07.136560 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:15:07.137464 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:15:07.137464 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:15:07.137464 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:15:07.137464 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:15:07.137464 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:07.137464 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:07.137464 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:07.137464 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:07.137464 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:07.137464 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:07.137464 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:07.137464 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:07.137464 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:07.137464 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:07.137464 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:07.137464 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:07.138310 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:07.138310 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:07.138310 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:07.138310 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:07.138310 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:15:07.138310 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:15:07.138310 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:15:07.138310 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:15:07.138310 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:15:07.138310 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:15:07.138310 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 19:15:07.138310 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:07.140981 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 237/479] 10.222.0.33 () {56 vars in 1114 bytes} [Fri Jul 15 19:15:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:07.146290 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412369f0-2678-4266-9b52-b0cc5b71ef23 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:07.262420 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412369f0-2678-4266-9b52-b0cc5b71ef23 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:07.294534 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:07.304870 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9b4902-3618-41e7-aa3a-4efe667bf329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:07.305558 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b4902-3618-41e7-aa3a-4efe667bf329 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:07.309253 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b4902-3618-41e7-aa3a-4efe667bf329 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:07.309253 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b4902-3618-41e7-aa3a-4efe667bf329 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:07.309253 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9b4902-3618-41e7-aa3a-4efe667bf329 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:07.309253 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9b4902-3618-41e7-aa3a-4efe667bf329 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:07.309253 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9b4902-3618-41e7-aa3a-4efe667bf329 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:07.320652 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 243/480] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 15 19:15:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:15:08.467020 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3374030-d854-4e3e-8cdc-6750b900fd6f None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:08.468069 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3374030-d854-4e3e-8cdc-6750b900fd6f None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:08.468580 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3374030-d854-4e3e-8cdc-6750b900fd6f None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:08.470259 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 238/481] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:15:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:15:08.476079 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1140a980-0ab0-4e42-bd36-ee779a625334 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:08.477199 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1140a980-0ab0-4e42-bd36-ee779a625334 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:08.477753 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1140a980-0ab0-4e42-bd36-ee779a625334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:08.502761 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-1140a980-0ab0-4e42-bd36-ee779a625334 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:15:08.504289 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-1140a980-0ab0-4e42-bd36-ee779a625334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:15:08.592375 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 244/482] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:15:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:15:08.627149 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3374030-d854-4e3e-8cdc-6750b900fd6f None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:08.643882 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42f8eb4-7e99-4f07-a3d2-dca835040702 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:08.644288 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42f8eb4-7e99-4f07-a3d2-dca835040702 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:08.644569 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42f8eb4-7e99-4f07-a3d2-dca835040702 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:08.646400 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 245/483] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:15:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:15:08.662646 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230c6dd6-d7cd-4608-b6e1-786e619ee863 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:08.662927 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230c6dd6-d7cd-4608-b6e1-786e619ee863 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:08.663202 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230c6dd6-d7cd-4608-b6e1-786e619ee863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:08.690443 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-230c6dd6-d7cd-4608-b6e1-786e619ee863 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:15:08.691635 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-230c6dd6-d7cd-4608-b6e1-786e619ee863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:15:08.810489 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3374030-d854-4e3e-8cdc-6750b900fd6f None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:08.829100 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 246/484] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:15:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:15:08.844652 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:08.856242 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e3f093-ca52-417a-bc59-86984dd90e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:08.857067 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e3f093-ca52-417a-bc59-86984dd90e7c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:08.857341 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e3f093-ca52-417a-bc59-86984dd90e7c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:08.857614 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e3f093-ca52-417a-bc59-86984dd90e7c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:08.868608 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e3f093-ca52-417a-bc59-86984dd90e7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:15:08.876266 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-23e3f093-ca52-417a-bc59-86984dd90e7c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:15:08.876266 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:08.876266 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:08.876266 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:08.876266 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:08.876266 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:08.876266 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:08.876266 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:08.876266 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:08.876266 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:08.876266 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:08.876266 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:08.876266 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:15:08.876266 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:15:08.876266 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:15:08.876266 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:15:08.877159 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:15:08.877159 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:15:08.877159 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:15:08.877159 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:15:08.877159 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:08.877159 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:08.877159 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:08.877159 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:08.877159 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:08.877159 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:08.877159 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:08.877159 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:08.877159 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:08.877159 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:08.877159 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:08.877159 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:08.878011 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:08.878011 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:08.878011 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:08.878011 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:08.878011 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:15:08.878011 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:15:08.878011 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:15:08.878011 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:15:08.878011 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:15:08.878011 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:15:08.878011 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:15:08.878011 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:08.881791 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230c6dd6-d7cd-4608-b6e1-786e619ee863 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:08.883061 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 239/485] 10.222.0.33 () {56 vars in 1102 bytes} [Fri Jul 15 19:15:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:08.889341 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e3f093-ca52-417a-bc59-86984dd90e7c None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:08.960559 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:09.015416 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230c6dd6-d7cd-4608-b6e1-786e619ee863 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:09.028653 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e3f093-ca52-417a-bc59-86984dd90e7c None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:09.059318 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ebd8bf-0c3c-4795-9498-4ca7fb042d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:09.060046 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ebd8bf-0c3c-4795-9498-4ca7fb042d8f None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:09.060377 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ebd8bf-0c3c-4795-9498-4ca7fb042d8f None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:09.060739 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ebd8bf-0c3c-4795-9498-4ca7fb042d8f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:09.073270 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ebd8bf-0c3c-4795-9498-4ca7fb042d8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:15:09.081680 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-03ebd8bf-0c3c-4795-9498-4ca7fb042d8f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:15:09.081680 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:09.081680 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:09.081680 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:09.081680 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:09.081680 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:09.081680 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:09.081680 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:09.081680 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:09.081680 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:09.081680 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:09.081680 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:09.081680 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:15:09.081680 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:15:09.081680 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:15:09.081680 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:15:09.082608 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:15:09.082608 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:15:09.082608 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:15:09.082608 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:15:09.082608 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:09.082608 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:09.082608 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:09.082608 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:09.082608 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:09.082608 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:09.082608 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:09.082608 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:09.082608 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:09.082608 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:09.082608 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:09.082608 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:09.083837 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:09.083837 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:09.083837 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:09.083837 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:09.083837 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:15:09.083837 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:15:09.083837 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:15:09.083837 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:15:09.083837 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:15:09.083837 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:15:09.083837 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:15:09.083837 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:09.083837 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 247/486] 10.222.0.33 () {56 vars in 1102 bytes} [Fri Jul 15 19:15:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:09.084845 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d237b3c0-6824-4738-ade2-4d644e05577e None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:09.084994 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d237b3c0-6824-4738-ade2-4d644e05577e None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:09.085262 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d237b3c0-6824-4738-ade2-4d644e05577e None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:09.086915 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 248/487] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:15:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:15:09.088160 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e147eb-7b85-4e51-85a6-898fb03d9eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:09.088586 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d237b3c0-6824-4738-ade2-4d644e05577e None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:09.089176 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e147eb-7b85-4e51-85a6-898fb03d9eeb None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:09.089599 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e147eb-7b85-4e51-85a6-898fb03d9eeb None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:09.090046 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e147eb-7b85-4e51-85a6-898fb03d9eeb None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:09.091043 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e147eb-7b85-4e51-85a6-898fb03d9eeb None admin] RBAC: Adding query filter params (name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:09.091519 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e147eb-7b85-4e51-85a6-898fb03d9eeb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:09.094436 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e147eb-7b85-4e51-85a6-898fb03d9eeb None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:09.111875 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 240/488] 10.222.0.33 () {56 vars in 1103 bytes} [Fri Jul 15 19:15:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:09.114017 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb9782c-7e31-444e-97d0-8812828381f0 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:09.114511 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb9782c-7e31-444e-97d0-8812828381f0 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:09.114988 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb9782c-7e31-444e-97d0-8812828381f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:09.137962 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-9fb9782c-7e31-444e-97d0-8812828381f0 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:15:09.139521 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9fb9782c-7e31-444e-97d0-8812828381f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:15:09.198849 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:09.210667 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d237b3c0-6824-4738-ade2-4d644e05577e None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:09.226890 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 241/489] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:15:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:15:09.227894 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb9782c-7e31-444e-97d0-8812828381f0 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:09.230026 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:09.256548 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c12374f-416f-4dc9-b469-35b78c07a6d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:09.258043 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12374f-416f-4dc9-b469-35b78c07a6d7 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:09.258608 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12374f-416f-4dc9-b469-35b78c07a6d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:09.259091 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12374f-416f-4dc9-b469-35b78c07a6d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:09.261592 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c12374f-416f-4dc9-b469-35b78c07a6d7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:09.262111 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c12374f-416f-4dc9-b469-35b78c07a6d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:09.263824 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c12374f-416f-4dc9-b469-35b78c07a6d7 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:09.275945 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 249/490] 10.222.0.33 () {56 vars in 1103 bytes} [Fri Jul 15 19:15:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:09.277327 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c12374f-416f-4dc9-b469-35b78c07a6d7 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:09.334059 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb9782c-7e31-444e-97d0-8812828381f0 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:09.374602 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09c20ad-ca12-46d4-8bdc-40d98ac7b0d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:09.375212 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09c20ad-ca12-46d4-8bdc-40d98ac7b0d0 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:09.375493 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09c20ad-ca12-46d4-8bdc-40d98ac7b0d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:09.380559 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09c20ad-ca12-46d4-8bdc-40d98ac7b0d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:09.388584 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09c20ad-ca12-46d4-8bdc-40d98ac7b0d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:09.390075 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09c20ad-ca12-46d4-8bdc-40d98ac7b0d0 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:09.409742 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-c09c20ad-ca12-46d4-8bdc-40d98ac7b0d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:09.409742 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:09.409742 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:09.409742 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:09.409742 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:09.409742 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:09.409742 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:09.409742 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:09.409742 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:09.409742 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:09.409742 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:09.409742 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:09.409742 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:15:09.409742 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:15:09.409742 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:15:09.410655 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:15:09.410655 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:09.410655 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:09.410655 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:09.410655 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:09.410655 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:09.410655 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:09.410655 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:09.410655 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:09.410655 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:09.410655 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:09.410655 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:09.410655 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:09.410655 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:09.410655 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:09.410655 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:09.410655 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:09.412107 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:15:09.412107 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:15:09.412107 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:09.412107 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:09.412107 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 242/491] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:15:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:09.412478 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09c20ad-ca12-46d4-8bdc-40d98ac7b0d0 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:09.417614 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c12374f-416f-4dc9-b469-35b78c07a6d7 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:09.442467 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:09.463890 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3fdf8d-35a8-48d4-ad44-8ed573203b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:09.468565 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3fdf8d-35a8-48d4-ad44-8ed573203b2d None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:09.468565 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3fdf8d-35a8-48d4-ad44-8ed573203b2d None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:09.468565 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3fdf8d-35a8-48d4-ad44-8ed573203b2d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:09.479344 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3fdf8d-35a8-48d4-ad44-8ed573203b2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:15:09.487528 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-0a3fdf8d-35a8-48d4-ad44-8ed573203b2d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:15:09.487528 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:09.487528 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:09.487528 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:09.487528 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:09.487528 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:09.487528 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:09.487528 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:09.487528 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:09.487528 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:09.487528 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:09.487528 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:09.487528 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:15:09.487528 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:15:09.487528 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:15:09.487528 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:15:09.488705 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:15:09.488705 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:15:09.488705 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:15:09.488705 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:15:09.488705 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:09.488705 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:09.488705 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:09.488705 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:09.488705 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:09.488705 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:09.488705 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:09.488705 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:09.488705 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:09.488705 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:09.488705 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:09.488705 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:09.489728 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:09.489728 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:09.489728 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:09.489728 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:09.489728 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:15:09.489728 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:15:09.489728 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:15:09.489728 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:15:09.489728 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:15:09.489728 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:15:09.489728 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:15:09.489728 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:09.492749 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 250/492] 10.222.0.33 () {56 vars in 1102 bytes} [Fri Jul 15 19:15:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:09.493281 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3fdf8d-35a8-48d4-ad44-8ed573203b2d None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:09.533593 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:09.550803 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09c20ad-ca12-46d4-8bdc-40d98ac7b0d0 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:09.600150 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e596323d-b302-4659-8d08-e032a0d6e829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:09.601452 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e596323d-b302-4659-8d08-e032a0d6e829 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:09.601907 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e596323d-b302-4659-8d08-e032a0d6e829 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:09.602559 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e596323d-b302-4659-8d08-e032a0d6e829 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:09.616523 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e596323d-b302-4659-8d08-e032a0d6e829 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:09.618966 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e596323d-b302-4659-8d08-e032a0d6e829 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:09.620451 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3fdf8d-35a8-48d4-ad44-8ed573203b2d None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:09.640982 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-e596323d-b302-4659-8d08-e032a0d6e829 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:09.640982 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:09.640982 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:09.640982 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:09.640982 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:09.640982 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:09.640982 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:09.640982 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:09.640982 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:09.640982 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:09.640982 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:09.640982 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:09.640982 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:15:09.640982 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:15:09.640982 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:15:09.642216 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:15:09.642216 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:09.642216 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:09.642216 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:09.642216 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:09.642216 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:09.642216 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:09.642216 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:09.642216 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:09.642216 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:09.642216 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:09.642216 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:09.642216 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:09.642216 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:09.642216 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:09.642216 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:09.642216 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:09.643059 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:15:09.643059 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:15:09.643059 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:09.643059 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:09.643059 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 243/493] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:15:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:09.643660 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e596323d-b302-4659-8d08-e032a0d6e829 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:09.661239 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0342c4-02f3-4fa1-a325-68d89c3e4c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:09.661941 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0342c4-02f3-4fa1-a325-68d89c3e4c45 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:09.662409 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0342c4-02f3-4fa1-a325-68d89c3e4c45 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:09.662684 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0342c4-02f3-4fa1-a325-68d89c3e4c45 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:09.663811 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0342c4-02f3-4fa1-a325-68d89c3e4c45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:09.664321 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0342c4-02f3-4fa1-a325-68d89c3e4c45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:09.665629 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0342c4-02f3-4fa1-a325-68d89c3e4c45 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:09.681640 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 251/494] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:15:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:09.681640 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada6b905-8f0e-4e5c-9a5c-d00dcaace5ef None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:09.681640 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada6b905-8f0e-4e5c-9a5c-d00dcaace5ef None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:09.681640 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada6b905-8f0e-4e5c-9a5c-d00dcaace5ef None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:09.681640 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 252/495] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:15:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:15:09.681640 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada6b905-8f0e-4e5c-9a5c-d00dcaace5ef None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:09.745892 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:09.766948 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e596323d-b302-4659-8d08-e032a0d6e829 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:09.793637 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:09.800277 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada6b905-8f0e-4e5c-9a5c-d00dcaace5ef None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:09.804152 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82344fc-e00c-4cc5-889a-8d125b44cc2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:09.804823 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82344fc-e00c-4cc5-889a-8d125b44cc2c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:09.805102 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82344fc-e00c-4cc5-889a-8d125b44cc2c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:09.805374 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82344fc-e00c-4cc5-889a-8d125b44cc2c None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:09.806155 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82344fc-e00c-4cc5-889a-8d125b44cc2c None admin] RBAC: Adding query filter params (name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:09.806457 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82344fc-e00c-4cc5-889a-8d125b44cc2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:09.807917 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82344fc-e00c-4cc5-889a-8d125b44cc2c None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:09.820980 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 244/496] 10.222.0.33 () {56 vars in 1103 bytes} [Fri Jul 15 19:15:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:09.820980 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82344fc-e00c-4cc5-889a-8d125b44cc2c None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:09.842952 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c9d7cc-4267-4ca7-ba57-149ded7ecdd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:09.843638 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c9d7cc-4267-4ca7-ba57-149ded7ecdd9 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:09.843915 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c9d7cc-4267-4ca7-ba57-149ded7ecdd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:09.844171 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c9d7cc-4267-4ca7-ba57-149ded7ecdd9 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:09.844889 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c9d7cc-4267-4ca7-ba57-149ded7ecdd9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:09.845201 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c9d7cc-4267-4ca7-ba57-149ded7ecdd9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:09.846608 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c9d7cc-4267-4ca7-ba57-149ded7ecdd9 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:09.857930 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 253/497] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:15:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:09.863500 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86fee82-e161-4873-b46c-b43bab0bb4f5 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:09.863790 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86fee82-e161-4873-b46c-b43bab0bb4f5 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:09.864085 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86fee82-e161-4873-b46c-b43bab0bb4f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:09.882161 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-a86fee82-e161-4873-b46c-b43bab0bb4f5 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:15:09.883361 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-a86fee82-e161-4873-b46c-b43bab0bb4f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:15:09.921845 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82344fc-e00c-4cc5-889a-8d125b44cc2c None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:09.961786 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 254/498] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:15:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:15:09.961786 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86fee82-e161-4873-b46c-b43bab0bb4f5 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:09.967046 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cd5a6f-aa47-46b9-ab9f-736273723b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:09.967928 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd5a6f-aa47-46b9-ab9f-736273723b18 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:09.968288 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd5a6f-aa47-46b9-ab9f-736273723b18 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:09.968572 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd5a6f-aa47-46b9-ab9f-736273723b18 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:09.976936 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cd5a6f-aa47-46b9-ab9f-736273723b18 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:09.978466 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cd5a6f-aa47-46b9-ab9f-736273723b18 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:09.996080 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-d0cd5a6f-aa47-46b9-ab9f-736273723b18 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 15 19:15:09.996080 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:09.996080 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:09.996080 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:09.996080 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:09.996080 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:09.996080 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:09.996080 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:09.996080 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:09.996080 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:09.996080 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:09.996080 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:09.996080 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 19:15:09.996080 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 19:15:09.996080 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 19:15:09.996080 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 19:15:09.997170 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:09.997170 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:09.997170 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:15:09.997170 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:15:09.997170 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:15:09.997170 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:15:09.997170 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:09.997170 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:09.997170 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:09.997170 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:09.997170 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:09.997170 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:09.997170 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:09.997170 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:09.997170 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:09.997170 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:09.997170 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:09.997170 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:09.998097 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:09.998097 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:09.998097 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 19:15:09.998097 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 19:15:09.998097 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 19:15:09.998097 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 19:15:09.998097 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 19:15:09.998097 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 19:15:09.998097 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 15 19:15:09.998097 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:09.998097 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 245/499] 10.222.0.33 () {56 vars in 1093 bytes} [Fri Jul 15 19:15:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:09.998097 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cd5a6f-aa47-46b9-ab9f-736273723b18 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:10.002431 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:10.052827 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:10.067856 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86fee82-e161-4873-b46c-b43bab0bb4f5 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:10.102289 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a74ebeb-2515-4ff0-954a-5f9c87e9085c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:10.103045 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cd5a6f-aa47-46b9-ab9f-736273723b18 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:10.103153 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a74ebeb-2515-4ff0-954a-5f9c87e9085c None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:10.103347 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a74ebeb-2515-4ff0-954a-5f9c87e9085c None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:10.103621 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a74ebeb-2515-4ff0-954a-5f9c87e9085c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:10.113624 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a74ebeb-2515-4ff0-954a-5f9c87e9085c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:10.115024 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a74ebeb-2515-4ff0-954a-5f9c87e9085c None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:10.133084 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-1a74ebeb-2515-4ff0-954a-5f9c87e9085c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:10.133084 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:10.133084 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:10.133084 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:10.133084 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:10.133084 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:10.133084 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:10.133084 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:10.133084 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:10.133084 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:10.133084 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:10.133084 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:10.133084 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:15:10.133084 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:15:10.133084 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:15:10.133997 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:15:10.133997 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:10.133997 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:10.133997 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:10.133997 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:10.133997 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:10.133997 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:10.133997 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:10.133997 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:10.133997 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:10.133997 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:10.133997 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:10.133997 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:10.133997 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:10.133997 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:10.133997 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:10.133997 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:10.134862 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:15:10.134862 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:15:10.134862 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:10.134862 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:10.134862 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 255/500] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:15:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:10.135124 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a74ebeb-2515-4ff0-954a-5f9c87e9085c None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:10.137475 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08258e34-828c-4531-bf50-df8476db6335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:10.138299 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08258e34-828c-4531-bf50-df8476db6335 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:10.138603 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08258e34-828c-4531-bf50-df8476db6335 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:10.138953 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08258e34-828c-4531-bf50-df8476db6335 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:10.148039 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08258e34-828c-4531-bf50-df8476db6335 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:10.149506 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08258e34-828c-4531-bf50-df8476db6335 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:10.163729 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-08258e34-828c-4531-bf50-df8476db6335 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 15 19:15:10.163729 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:10.163729 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:10.163729 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:10.163729 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:10.163729 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:10.163729 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:10.163729 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:10.163729 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:10.163729 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:10.163729 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:10.163729 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:10.163729 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 19:15:10.163729 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 19:15:10.163729 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 19:15:10.163729 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 19:15:10.164751 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:10.164751 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:10.164751 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:15:10.164751 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:15:10.164751 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:15:10.164751 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:15:10.164751 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:10.164751 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:10.164751 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:10.164751 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:10.164751 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:10.164751 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:10.164751 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:10.164751 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:10.164751 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:10.164751 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:10.164751 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:10.164751 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:10.165668 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:10.165668 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:10.165668 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 19:15:10.165668 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 19:15:10.165668 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 19:15:10.165668 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 19:15:10.165668 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 19:15:10.165668 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 19:15:10.165668 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 15 19:15:10.165668 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:10.165668 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 246/501] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 15 19:15:09 2022] GET /identity/v3/users/glance => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:10.165668 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08258e34-828c-4531-bf50-df8476db6335 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:10.225371 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:10.240406 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a74ebeb-2515-4ff0-954a-5f9c87e9085c None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:10.262405 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08258e34-828c-4531-bf50-df8476db6335 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:10.274808 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a21a383-dff3-4581-887d-bc34f0d8b2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:10.275525 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a21a383-dff3-4581-887d-bc34f0d8b2f4 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:10.275805 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a21a383-dff3-4581-887d-bc34f0d8b2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:10.276077 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a21a383-dff3-4581-887d-bc34f0d8b2f4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:10.282991 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:10.286535 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a21a383-dff3-4581-887d-bc34f0d8b2f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:15:10.294158 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-8a21a383-dff3-4581-887d-bc34f0d8b2f4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:15:10.294158 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:10.294158 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:10.294158 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:10.294158 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:10.294158 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:10.294158 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:10.294158 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:10.294158 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:10.294158 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:10.294158 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:10.294158 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:10.294158 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:15:10.294158 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:15:10.294158 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:15:10.294158 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:15:10.301230 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:15:10.301230 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:15:10.301230 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:15:10.301230 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:15:10.301230 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:10.301230 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:10.301230 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:10.301230 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:10.301230 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:10.301230 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:10.301230 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:10.301230 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:10.301230 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:10.301230 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:10.301230 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:10.301230 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:10.302269 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:10.302269 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:10.302269 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:10.302269 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:10.302269 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:15:10.302269 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:15:10.302269 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:15:10.302269 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:15:10.302269 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:15:10.302269 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:15:10.302269 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:15:10.302269 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:10.302269 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 256/502] 10.222.0.33 () {56 vars in 1102 bytes} [Fri Jul 15 19:15:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:10.302269 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8052fe4f-d1ae-43d4-8468-24ca74592429 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:10.303127 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8052fe4f-d1ae-43d4-8468-24ca74592429 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:10.303127 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8052fe4f-d1ae-43d4-8468-24ca74592429 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:10.303256 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe0ebec-4c6c-4afe-8368-e4709f89e3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:10.303256 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe0ebec-4c6c-4afe-8368-e4709f89e3c9 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:10.303256 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe0ebec-4c6c-4afe-8368-e4709f89e3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:10.303455 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 257/503] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:15:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:15:10.303531 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe0ebec-4c6c-4afe-8368-e4709f89e3c9 None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:10.303600 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8052fe4f-d1ae-43d4-8468-24ca74592429 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:10.303666 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe0ebec-4c6c-4afe-8368-e4709f89e3c9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:10.303666 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe0ebec-4c6c-4afe-8368-e4709f89e3c9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:10.303666 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe0ebec-4c6c-4afe-8368-e4709f89e3c9 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:10.337368 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 247/504] 10.222.0.33 () {56 vars in 1133 bytes} [Fri Jul 15 19:15:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:10.338241 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe0ebec-4c6c-4afe-8368-e4709f89e3c9 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:10.408579 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8052fe4f-d1ae-43d4-8468-24ca74592429 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:10.442720 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe0ebec-4c6c-4afe-8368-e4709f89e3c9 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:10.443298 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4908aa1-0725-4b4d-8a0b-b67f486d572a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:10.443998 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4908aa1-0725-4b4d-8a0b-b67f486d572a None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:10.444282 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4908aa1-0725-4b4d-8a0b-b67f486d572a None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:10.444555 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4908aa1-0725-4b4d-8a0b-b67f486d572a None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:10.445278 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4908aa1-0725-4b4d-8a0b-b67f486d572a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:10.445618 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4908aa1-0725-4b4d-8a0b-b67f486d572a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:10.447105 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4908aa1-0725-4b4d-8a0b-b67f486d572a None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:10.458693 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 258/505] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:15:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:10.459499 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4908aa1-0725-4b4d-8a0b-b67f486d572a None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:10.464063 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:10.477885 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684bc3b4-1bad-41e2-820a-f42972a91e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:10.478551 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684bc3b4-1bad-41e2-820a-f42972a91e94 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:10.478832 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684bc3b4-1bad-41e2-820a-f42972a91e94 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:10.479110 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684bc3b4-1bad-41e2-820a-f42972a91e94 None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:10.479940 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684bc3b4-1bad-41e2-820a-f42972a91e94 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:10.480244 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684bc3b4-1bad-41e2-820a-f42972a91e94 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:10.481687 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684bc3b4-1bad-41e2-820a-f42972a91e94 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:10.500368 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 248/506] 10.222.0.33 () {56 vars in 1137 bytes} [Fri Jul 15 19:15:10 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:10.501569 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be70ab9-1282-4b51-b923-7ef0c8b0d072 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:10.501886 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be70ab9-1282-4b51-b923-7ef0c8b0d072 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:10.502160 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be70ab9-1282-4b51-b923-7ef0c8b0d072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:10.517685 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-5be70ab9-1282-4b51-b923-7ef0c8b0d072 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:15:10.518910 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-5be70ab9-1282-4b51-b923-7ef0c8b0d072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:15:10.523347 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:10.565831 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4908aa1-0725-4b4d-8a0b-b67f486d572a None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:10.586918 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 249/507] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:15:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:15:10.588284 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be70ab9-1282-4b51-b923-7ef0c8b0d072 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:10.599184 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f16ec0-54a8-4e49-823c-68a30f769460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:10.600340 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f16ec0-54a8-4e49-823c-68a30f769460 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:10.600598 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f16ec0-54a8-4e49-823c-68a30f769460 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:10.600876 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f16ec0-54a8-4e49-823c-68a30f769460 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:10.601621 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f16ec0-54a8-4e49-823c-68a30f769460 None admin] RBAC: Adding query filter params (name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:10.601958 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f16ec0-54a8-4e49-823c-68a30f769460 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:10.603321 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f16ec0-54a8-4e49-823c-68a30f769460 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:10.611397 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 259/508] 10.222.0.33 () {56 vars in 1103 bytes} [Fri Jul 15 19:15:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:10.612196 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f16ec0-54a8-4e49-823c-68a30f769460 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:10.693389 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be70ab9-1282-4b51-b923-7ef0c8b0d072 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:10.702808 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:10.712727 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f16ec0-54a8-4e49-823c-68a30f769460 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:10.728919 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e20010d-ff09-4bed-a864-128a2e26f48f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:10.729574 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e20010d-ff09-4bed-a864-128a2e26f48f None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:10.729890 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e20010d-ff09-4bed-a864-128a2e26f48f None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:10.730171 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e20010d-ff09-4bed-a864-128a2e26f48f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:10.737086 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:10.750478 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6d2ace-e06f-4a4c-a70b-0518d0889923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:10.751243 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6d2ace-e06f-4a4c-a70b-0518d0889923 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:10.751523 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6d2ace-e06f-4a4c-a70b-0518d0889923 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:10.751796 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6d2ace-e06f-4a4c-a70b-0518d0889923 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:10.759175 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6d2ace-e06f-4a4c-a70b-0518d0889923 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:10.760603 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6d2ace-e06f-4a4c-a70b-0518d0889923 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:10.776512 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-da6d2ace-e06f-4a4c-a70b-0518d0889923 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 15 19:15:10.776512 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:10.776512 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:10.776512 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:10.776512 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:10.776512 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:10.776512 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:10.776512 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:10.776512 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:10.776512 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:10.776512 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:10.776512 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:10.776512 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 19:15:10.776512 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 19:15:10.776512 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 19:15:10.776512 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 19:15:10.777453 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:10.777453 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:10.777453 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:15:10.777453 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:15:10.777453 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:15:10.777453 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:15:10.777453 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:10.777453 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:10.777453 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:10.777453 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:10.777453 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:10.777453 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:10.777453 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:10.777453 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:10.777453 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:10.777453 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:10.777453 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:10.777453 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:10.778399 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:10.778399 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:10.778399 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 19:15:10.778399 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 19:15:10.778399 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 19:15:10.778399 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 19:15:10.778399 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 19:15:10.778399 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 19:15:10.778399 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 15 19:15:10.778399 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:10.778399 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 260/509] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 15 19:15:10 2022] GET /identity/v3/users/cinder => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:10.778977 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6d2ace-e06f-4a4c-a70b-0518d0889923 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:10.803479 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e20010d-ff09-4bed-a864-128a2e26f48f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:10.805012 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e20010d-ff09-4bed-a864-128a2e26f48f None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:10.878945 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6d2ace-e06f-4a4c-a70b-0518d0889923 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:10.903566 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 250/510] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 15 19:15:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 19:15:10.904416 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e20010d-ff09-4bed-a864-128a2e26f48f None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:10.915107 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:10.917322 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d23b6e-fda0-4938-8fd5-35d3d6a045b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:10.918091 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d23b6e-fda0-4938-8fd5-35d3d6a045b8 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:10.918373 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d23b6e-fda0-4938-8fd5-35d3d6a045b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:10.918645 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d23b6e-fda0-4938-8fd5-35d3d6a045b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:10.972762 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:10.996271 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d23b6e-fda0-4938-8fd5-35d3d6a045b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:10.997657 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d23b6e-fda0-4938-8fd5-35d3d6a045b8 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:11.019903 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e20010d-ff09-4bed-a864-128a2e26f48f None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:11.056969 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58b84a4-7a2a-41bf-a247-75769c019323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:11.058163 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58b84a4-7a2a-41bf-a247-75769c019323 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:11.058440 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58b84a4-7a2a-41bf-a247-75769c019323 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:11.058711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58b84a4-7a2a-41bf-a247-75769c019323 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:11.069566 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58b84a4-7a2a-41bf-a247-75769c019323 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:11.071232 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58b84a4-7a2a-41bf-a247-75769c019323 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:11.091269 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-a58b84a4-7a2a-41bf-a247-75769c019323 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:11.091269 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:11.091269 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:11.091269 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:11.091269 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:11.091269 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:11.091269 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:11.091269 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:11.091269 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:11.091269 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:11.091269 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:11.091269 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:11.091269 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:15:11.091269 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:15:11.091269 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:15:11.092936 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:15:11.092936 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:11.092936 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:11.092936 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:11.092936 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:11.092936 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:11.092936 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:11.092936 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:11.092936 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:11.092936 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:11.092936 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:11.092936 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:11.092936 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:11.092936 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:11.092936 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:11.092936 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:11.092936 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:11.094978 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:15:11.094978 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:15:11.094978 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:11.094978 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:11.094978 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 251/511] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:15:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:11.094978 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58b84a4-7a2a-41bf-a247-75769c019323 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:11.109797 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 261/512] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 15 19:15:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 19:15:11.110785 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d23b6e-fda0-4938-8fd5-35d3d6a045b8 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:11.156284 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:11.196383 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:11.198031 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58b84a4-7a2a-41bf-a247-75769c019323 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:11.215879 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d23b6e-fda0-4938-8fd5-35d3d6a045b8 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:11.232949 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6b4f5f-4e1b-4eda-acfa-85ca009b42f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:11.233670 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6b4f5f-4e1b-4eda-acfa-85ca009b42f0 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:11.233975 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6b4f5f-4e1b-4eda-acfa-85ca009b42f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:11.234246 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6b4f5f-4e1b-4eda-acfa-85ca009b42f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:11.244096 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6b4f5f-4e1b-4eda-acfa-85ca009b42f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:11.245551 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6b4f5f-4e1b-4eda-acfa-85ca009b42f0 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:11.249900 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa58de83-a055-41be-a20b-1bec6f4434b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:11.250560 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa58de83-a055-41be-a20b-1bec6f4434b0 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:11.250835 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa58de83-a055-41be-a20b-1bec6f4434b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:11.251106 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa58de83-a055-41be-a20b-1bec6f4434b0 None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:11.251938 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa58de83-a055-41be-a20b-1bec6f4434b0 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:11.252242 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa58de83-a055-41be-a20b-1bec6f4434b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:11.253711 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa58de83-a055-41be-a20b-1bec6f4434b0 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:11.263021 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-8c6b4f5f-4e1b-4eda-acfa-85ca009b42f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:11.263021 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:11.263021 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:11.263021 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:11.263021 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:11.263021 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:11.263021 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:11.263021 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:11.263021 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:11.263021 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:11.263021 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:11.263021 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:11.263021 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:15:11.263021 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:15:11.263021 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:15:11.264052 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:15:11.264052 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:11.264052 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:11.264052 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:11.264052 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:11.264052 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:11.264052 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:11.264052 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:11.264052 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:11.264052 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:11.264052 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:11.264052 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:11.264052 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:11.264052 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:11.264052 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:11.264052 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:11.264052 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:11.264949 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:15:11.264949 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:15:11.264949 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:11.264949 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:11.264949 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 252/513] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:15:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:11.265399 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6b4f5f-4e1b-4eda-acfa-85ca009b42f0 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:11.297171 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 262/514] 10.222.0.33 () {56 vars in 1137 bytes} [Fri Jul 15 19:15:11 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:11.298148 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa58de83-a055-41be-a20b-1bec6f4434b0 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:11.380778 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6b4f5f-4e1b-4eda-acfa-85ca009b42f0 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:11.394993 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:11.406692 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa58de83-a055-41be-a20b-1bec6f4434b0 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:11.419551 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc607695-796b-4dcd-aa3f-dfd7ceeac0e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:11.420462 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc607695-796b-4dcd-aa3f-dfd7ceeac0e3 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:11.420741 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc607695-796b-4dcd-aa3f-dfd7ceeac0e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:11.421022 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc607695-796b-4dcd-aa3f-dfd7ceeac0e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:11.432523 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc607695-796b-4dcd-aa3f-dfd7ceeac0e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:11.434028 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc607695-796b-4dcd-aa3f-dfd7ceeac0e3 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:11.441062 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9374134-c67c-4c34-af2a-1f0d7cf9af71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:11.441726 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9374134-c67c-4c34-af2a-1f0d7cf9af71 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:11.442034 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9374134-c67c-4c34-af2a-1f0d7cf9af71 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:11.442308 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9374134-c67c-4c34-af2a-1f0d7cf9af71 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:11.443035 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9374134-c67c-4c34-af2a-1f0d7cf9af71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:11.443341 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9374134-c67c-4c34-af2a-1f0d7cf9af71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:11.445286 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9374134-c67c-4c34-af2a-1f0d7cf9af71 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:11.451962 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:11.452527 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-fc607695-796b-4dcd-aa3f-dfd7ceeac0e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:11.452527 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:11.452527 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:11.452527 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:11.452527 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:11.452527 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:11.452527 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:11.452527 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:11.452527 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:11.452527 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:11.452527 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:11.452527 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:11.452527 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:15:11.452527 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:15:11.452527 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:15:11.453492 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:15:11.453492 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:11.453492 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:11.453492 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:11.453492 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:11.453492 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:11.453492 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:11.453492 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:11.453492 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:11.453492 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:11.453492 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:11.453492 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:11.453492 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:11.453492 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:11.453492 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:11.453492 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:11.453492 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:11.454621 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:15:11.454621 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:15:11.454621 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:11.454621 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:11.454621 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 253/515] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:15:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:11.454621 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc607695-796b-4dcd-aa3f-dfd7ceeac0e3 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:11.463183 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 263/516] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:15:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:11.463183 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9374134-c67c-4c34-af2a-1f0d7cf9af71 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:11.562218 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9374134-c67c-4c34-af2a-1f0d7cf9af71 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:11.563368 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc607695-796b-4dcd-aa3f-dfd7ceeac0e3 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:11.606135 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c725fe3-240b-4194-8d3c-c05e838ba717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:11.607122 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c725fe3-240b-4194-8d3c-c05e838ba717 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:11.607410 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c725fe3-240b-4194-8d3c-c05e838ba717 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:11.607914 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c725fe3-240b-4194-8d3c-c05e838ba717 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:11.608595 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c725fe3-240b-4194-8d3c-c05e838ba717 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:11.608956 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c725fe3-240b-4194-8d3c-c05e838ba717 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:11.610344 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c725fe3-240b-4194-8d3c-c05e838ba717 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:11.610856 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e47e1eb-d84b-479f-b679-9c8a744358dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:11.611669 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e47e1eb-d84b-479f-b679-9c8a744358dd None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:11.611938 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e47e1eb-d84b-479f-b679-9c8a744358dd None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:11.612210 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e47e1eb-d84b-479f-b679-9c8a744358dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:11.623009 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e47e1eb-d84b-479f-b679-9c8a744358dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:11.623188 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 264/517] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:15:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:11.623985 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c725fe3-240b-4194-8d3c-c05e838ba717 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:11.624444 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e47e1eb-d84b-479f-b679-9c8a744358dd None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:11.643103 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-2e47e1eb-d84b-479f-b679-9c8a744358dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:11.643103 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:11.643103 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:11.643103 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:11.643103 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:11.643103 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:11.643103 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:11.643103 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:11.643103 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:11.643103 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:11.643103 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:11.643103 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:11.643103 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:15:11.643103 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:15:11.643103 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:15:11.644148 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:15:11.644148 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:11.644148 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:11.644148 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:11.644148 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:11.644148 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:11.644148 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:11.644148 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:11.644148 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:11.644148 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:11.644148 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:11.644148 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:11.644148 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:11.644148 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:11.644148 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:11.644148 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:11.644148 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:11.645048 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:15:11.645048 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:15:11.645048 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:11.645048 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:11.645048 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 254/518] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:15:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:11.645344 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e47e1eb-d84b-479f-b679-9c8a744358dd None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:11.662747 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:11.686342 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:11.727742 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c725fe3-240b-4194-8d3c-c05e838ba717 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:11.744217 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e47e1eb-d84b-479f-b679-9c8a744358dd None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:11.769269 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9417b42-ef42-4d85-9e17-c073bc910f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:11.770035 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9417b42-ef42-4d85-9e17-c073bc910f29 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:11.770321 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9417b42-ef42-4d85-9e17-c073bc910f29 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:11.770597 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9417b42-ef42-4d85-9e17-c073bc910f29 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:11.782277 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636294e1-6447-4264-a1bb-7827966dce6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:11.782938 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636294e1-6447-4264-a1bb-7827966dce6c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:11.783217 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636294e1-6447-4264-a1bb-7827966dce6c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:11.783491 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636294e1-6447-4264-a1bb-7827966dce6c None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:11.784211 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636294e1-6447-4264-a1bb-7827966dce6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:11.784513 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636294e1-6447-4264-a1bb-7827966dce6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:11.785922 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636294e1-6447-4264-a1bb-7827966dce6c None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:11.797644 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 255/519] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:15:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:11.798538 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636294e1-6447-4264-a1bb-7827966dce6c None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:11.839200 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9417b42-ef42-4d85-9e17-c073bc910f29 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:11.840591 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9417b42-ef42-4d85-9e17-c073bc910f29 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:11.874176 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:11.896299 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636294e1-6447-4264-a1bb-7827966dce6c None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:11.899654 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:11.930249 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df29d07-13a4-45b4-8b05-6e0738f551ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:11.930906 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df29d07-13a4-45b4-8b05-6e0738f551ca None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:11.931183 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df29d07-13a4-45b4-8b05-6e0738f551ca None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:11.931453 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df29d07-13a4-45b4-8b05-6e0738f551ca None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:11.932147 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df29d07-13a4-45b4-8b05-6e0738f551ca None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:11.933576 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df29d07-13a4-45b4-8b05-6e0738f551ca None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:11.940085 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 265/520] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 15 19:15:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 19:15:11.940951 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9417b42-ef42-4d85-9e17-c073bc910f29 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:12.027307 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 256/521] 10.222.0.33 () {60 vars in 1132 bytes} [Fri Jul 15 19:15:11 2022] POST /identity/v3/users => generated 247 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:15:12.028133 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df29d07-13a4-45b4-8b05-6e0738f551ca None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:12.044676 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9417b42-ef42-4d85-9e17-c073bc910f29 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:12.087055 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f237ad1-34c2-490a-9dda-c80d4d659be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:12.087860 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f237ad1-34c2-490a-9dda-c80d4d659be6 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:12.088134 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f237ad1-34c2-490a-9dda-c80d4d659be6 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:12.088403 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f237ad1-34c2-490a-9dda-c80d4d659be6 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:12.096549 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f237ad1-34c2-490a-9dda-c80d4d659be6 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:12.098020 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f237ad1-34c2-490a-9dda-c80d4d659be6 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:12.113410 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-8f237ad1-34c2-490a-9dda-c80d4d659be6 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 15 19:15:12.113410 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:12.113410 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:12.113410 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:12.113410 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:12.113410 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:12.113410 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:12.113410 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:12.113410 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:12.113410 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:12.113410 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:12.113410 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:12.113410 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 19:15:12.113410 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 19:15:12.113410 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 19:15:12.113410 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 19:15:12.114645 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:12.114645 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:12.114645 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:15:12.114645 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:15:12.114645 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:15:12.114645 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:15:12.114645 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:12.114645 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:12.114645 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:12.114645 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:12.114645 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:12.114645 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:12.114645 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:12.114645 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:12.114645 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:12.114645 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:12.114645 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:12.114645 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:12.116510 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:12.116510 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:12.116510 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 19:15:12.116510 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 19:15:12.116510 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 19:15:12.116510 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 19:15:12.116510 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 19:15:12.116510 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 19:15:12.116510 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 15 19:15:12.116510 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:12.119992 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 266/522] 10.222.0.33 () {56 vars in 1102 bytes} [Fri Jul 15 19:15:11 2022] GET /identity/v3/users/neutron => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:12.120813 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f237ad1-34c2-490a-9dda-c80d4d659be6 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:12.141209 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:12.149825 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df29d07-13a4-45b4-8b05-6e0738f551ca None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:12.186984 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:12.193190 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f2317b-415e-4475-94f3-3c5b2c470222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:12.193644 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f2317b-415e-4475-94f3-3c5b2c470222 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:12.194040 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f2317b-415e-4475-94f3-3c5b2c470222 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:12.194314 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f2317b-415e-4475-94f3-3c5b2c470222 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:12.195054 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f2317b-415e-4475-94f3-3c5b2c470222 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:12.195365 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f2317b-415e-4475-94f3-3c5b2c470222 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:12.196740 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f2317b-415e-4475-94f3-3c5b2c470222 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:12.209857 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 257/523] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:15:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:12.211151 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f2317b-415e-4475-94f3-3c5b2c470222 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:12.237597 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f237ad1-34c2-490a-9dda-c80d4d659be6 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:12.274626 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3603c495-46ae-4eae-a900-f631f204ae76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:12.275375 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3603c495-46ae-4eae-a900-f631f204ae76 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:12.275666 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3603c495-46ae-4eae-a900-f631f204ae76 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:12.275954 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3603c495-46ae-4eae-a900-f631f204ae76 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:12.283060 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3603c495-46ae-4eae-a900-f631f204ae76 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:12.284494 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3603c495-46ae-4eae-a900-f631f204ae76 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:12.298925 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-3603c495-46ae-4eae-a900-f631f204ae76 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:15:12.298925 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:12.298925 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:12.298925 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:12.298925 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:12.298925 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:12.298925 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:12.298925 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:12.298925 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:12.298925 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:12.298925 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:12.298925 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:12.298925 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 19:15:12.298925 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 19:15:12.298925 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 19:15:12.299925 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 19:15:12.299925 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:12.299925 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:12.299925 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:12.299925 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:12.299925 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:12.299925 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:12.299925 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:12.299925 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:12.299925 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:12.299925 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:12.299925 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:12.299925 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:12.299925 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:12.299925 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:12.299925 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:12.299925 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:12.301594 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 19:15:12.301594 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 19:15:12.301594 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:15:12.301594 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:15:12.301594 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:15:12.301594 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:15:12.301594 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:15:12.301594 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:12.301594 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 267/524] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 15 19:15:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:12.301594 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3603c495-46ae-4eae-a900-f631f204ae76 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:12.329985 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f2317b-415e-4475-94f3-3c5b2c470222 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:12.368705 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88829b84-2781-4883-87c2-8501acd63e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:12.369538 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88829b84-2781-4883-87c2-8501acd63e4d None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:12.369848 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88829b84-2781-4883-87c2-8501acd63e4d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:12.370243 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88829b84-2781-4883-87c2-8501acd63e4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:12.382022 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88829b84-2781-4883-87c2-8501acd63e4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:12.382598 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:12.384475 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88829b84-2781-4883-87c2-8501acd63e4d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:12.404608 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-88829b84-2781-4883-87c2-8501acd63e4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:12.404608 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:12.404608 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:12.404608 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:12.404608 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:12.404608 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:12.404608 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:12.404608 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:12.404608 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:12.404608 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:12.404608 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:12.404608 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:12.404608 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:15:12.404608 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:15:12.404608 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:15:12.405534 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:15:12.405534 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:12.405534 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:12.405534 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:12.405534 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:12.405534 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:12.405534 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:12.405534 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:12.405534 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:12.405534 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:12.405534 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:12.405534 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:12.405534 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:12.405534 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:12.405534 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:12.405534 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:12.405534 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:12.406850 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:15:12.406850 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:15:12.406850 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:12.406850 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:12.407075 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3603c495-46ae-4eae-a900-f631f204ae76 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:12.410072 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 258/525] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:15:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:12.411104 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88829b84-2781-4883-87c2-8501acd63e4d None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:12.442871 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ba79d6-643d-461e-815d-9b67fd55b3b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:12.443590 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ba79d6-643d-461e-815d-9b67fd55b3b1 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:12.443871 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ba79d6-643d-461e-815d-9b67fd55b3b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:12.444141 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ba79d6-643d-461e-815d-9b67fd55b3b1 None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:12.444972 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ba79d6-643d-461e-815d-9b67fd55b3b1 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:12.445297 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ba79d6-643d-461e-815d-9b67fd55b3b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:12.446998 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ba79d6-643d-461e-815d-9b67fd55b3b1 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:12.457625 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:12.466489 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 268/526] 10.222.0.33 () {56 vars in 1139 bytes} [Fri Jul 15 19:15:12 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:12.471686 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ba79d6-643d-461e-815d-9b67fd55b3b1 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:12.512603 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88829b84-2781-4883-87c2-8501acd63e4d None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:12.550886 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0757450e-10db-415a-8000-544300af97c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:12.551686 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0757450e-10db-415a-8000-544300af97c6 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:12.551973 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0757450e-10db-415a-8000-544300af97c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:12.552256 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0757450e-10db-415a-8000-544300af97c6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:12.559319 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0757450e-10db-415a-8000-544300af97c6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:12.560777 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0757450e-10db-415a-8000-544300af97c6 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:12.575890 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-0757450e-10db-415a-8000-544300af97c6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:15:12.575890 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:12.575890 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:12.575890 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:12.575890 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:12.575890 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:12.575890 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:12.575890 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:12.575890 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:12.575890 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:12.575890 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:12.575890 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:12.575890 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 19:15:12.575890 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 19:15:12.575890 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 19:15:12.576813 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 19:15:12.576813 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:12.576813 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:12.576813 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:12.576813 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:12.576813 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:12.576813 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:12.576813 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:12.576813 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:12.576813 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:12.576813 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:12.576813 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:12.576813 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:12.576813 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:12.576813 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:12.576813 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:12.576813 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:12.577674 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 19:15:12.577674 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 19:15:12.577674 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:15:12.577674 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:15:12.577674 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:15:12.577674 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:15:12.577674 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:15:12.577674 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:12.577674 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 259/527] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 15 19:15:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:12.578188 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0757450e-10db-415a-8000-544300af97c6 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:12.585930 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ba79d6-643d-461e-815d-9b67fd55b3b1 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:12.621412 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f1ad05-114d-4706-9186-b199b5e5ed7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:12.622035 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f1ad05-114d-4706-9186-b199b5e5ed7f None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:12.622311 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f1ad05-114d-4706-9186-b199b5e5ed7f None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:12.622583 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f1ad05-114d-4706-9186-b199b5e5ed7f None admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:12.623369 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f1ad05-114d-4706-9186-b199b5e5ed7f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:12.623672 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f1ad05-114d-4706-9186-b199b5e5ed7f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:12.625095 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f1ad05-114d-4706-9186-b199b5e5ed7f None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:12.629085 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:12.648922 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 269/528] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jul 15 19:15:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:12.648922 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f1ad05-114d-4706-9186-b199b5e5ed7f None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:12.689256 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0757450e-10db-415a-8000-544300af97c6 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:12.692878 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:12.730535 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad0c555-0f3f-4419-9642-a70f2e8ef687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:12.731204 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad0c555-0f3f-4419-9642-a70f2e8ef687 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:12.731490 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad0c555-0f3f-4419-9642-a70f2e8ef687 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:12.731770 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad0c555-0f3f-4419-9642-a70f2e8ef687 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:12.732494 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad0c555-0f3f-4419-9642-a70f2e8ef687 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:12.732802 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad0c555-0f3f-4419-9642-a70f2e8ef687 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:12.734208 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad0c555-0f3f-4419-9642-a70f2e8ef687 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:12.751050 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 260/529] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:15:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:12.751050 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad0c555-0f3f-4419-9642-a70f2e8ef687 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:12.753591 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f1ad05-114d-4706-9186-b199b5e5ed7f None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:12.792431 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e366526-9216-41b3-8bde-89d6328fb247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:12.793361 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e366526-9216-41b3-8bde-89d6328fb247 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:12.793656 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e366526-9216-41b3-8bde-89d6328fb247 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:12.793968 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e366526-9216-41b3-8bde-89d6328fb247 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:12.861027 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:12.868927 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e366526-9216-41b3-8bde-89d6328fb247 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:12.870533 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e366526-9216-41b3-8bde-89d6328fb247 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:12.872800 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad0c555-0f3f-4419-9642-a70f2e8ef687 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:12.908236 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85586037-11a5-4acb-b7fe-29983362a10f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:12.908889 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85586037-11a5-4acb-b7fe-29983362a10f None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:12.909165 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85586037-11a5-4acb-b7fe-29983362a10f None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:12.909433 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85586037-11a5-4acb-b7fe-29983362a10f None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:12.910238 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85586037-11a5-4acb-b7fe-29983362a10f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:12.910539 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85586037-11a5-4acb-b7fe-29983362a10f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:12.911964 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85586037-11a5-4acb-b7fe-29983362a10f None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:12.930161 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 261/530] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jul 15 19:15:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:12.931316 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85586037-11a5-4acb-b7fe-29983362a10f None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:12.957409 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:12.985116 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 270/531] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 15 19:15:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 19:15:12.985116 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e366526-9216-41b3-8bde-89d6328fb247 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:13.043885 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85586037-11a5-4acb-b7fe-29983362a10f None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:13.079279 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93e7b20-efc0-4f0d-ac6b-7f44c979d7a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:13.080077 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93e7b20-efc0-4f0d-ac6b-7f44c979d7a6 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:13.080467 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93e7b20-efc0-4f0d-ac6b-7f44c979d7a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:13.080837 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93e7b20-efc0-4f0d-ac6b-7f44c979d7a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:13.081782 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93e7b20-efc0-4f0d-ac6b-7f44c979d7a6 None admin] RBAC: Adding query filter params (role.id=70de044d6962417593b6d03571e7502e, scope.project.id=55a756ccaef14cb49b26b0d68dd0b466, user.id=7673a3b5c0324e07a9535ac70ed7a491) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:13.082196 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93e7b20-efc0-4f0d-ac6b-7f44c979d7a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:13.083558 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:13.083819 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93e7b20-efc0-4f0d-ac6b-7f44c979d7a6 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:13.090129 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e366526-9216-41b3-8bde-89d6328fb247 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:13.102042 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 262/532] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 15 19:15:12 2022] GET /identity/v3/role_assignments?user.id=7673a3b5c0324e07a9535ac70ed7a491&scope.project.id=55a756ccaef14cb49b26b0d68dd0b466&role.id=70de044d6962417593b6d03571e7502e => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:13.102042 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93e7b20-efc0-4f0d-ac6b-7f44c979d7a6 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:13.141635 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb48c9fd-8c67-4e8a-bb66-afe6834e3d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:13.142508 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb48c9fd-8c67-4e8a-bb66-afe6834e3d2d None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:13.142868 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb48c9fd-8c67-4e8a-bb66-afe6834e3d2d None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:13.143218 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb48c9fd-8c67-4e8a-bb66-afe6834e3d2d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:13.152017 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb48c9fd-8c67-4e8a-bb66-afe6834e3d2d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:13.153615 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb48c9fd-8c67-4e8a-bb66-afe6834e3d2d None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:13.169685 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-fb48c9fd-8c67-4e8a-bb66-afe6834e3d2d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:15:13.169685 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:13.169685 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:13.169685 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:13.169685 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:13.169685 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:13.169685 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:13.169685 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:13.169685 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:13.169685 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:13.169685 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:13.169685 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:13.169685 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 19:15:13.169685 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 19:15:13.169685 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 19:15:13.170812 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 19:15:13.170812 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:13.170812 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:13.170812 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:13.170812 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:13.170812 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:13.170812 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:13.170812 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:13.170812 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:13.170812 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:13.170812 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:13.170812 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:13.170812 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:13.170812 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:13.170812 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:13.170812 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:13.170812 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:13.171888 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 19:15:13.171888 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 19:15:13.171888 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:15:13.171888 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:15:13.171888 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:15:13.171888 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:15:13.171888 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:15:13.171888 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:13.172938 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 271/533] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 15 19:15:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:13.173902 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb48c9fd-8c67-4e8a-bb66-afe6834e3d2d None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:13.226889 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:13.242940 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93e7b20-efc0-4f0d-ac6b-7f44c979d7a6 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:13.294876 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc8c76f-f9ba-4c69-bf3b-a016f3df16bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:13.295541 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc8c76f-f9ba-4c69-bf3b-a016f3df16bf None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:13.295822 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc8c76f-f9ba-4c69-bf3b-a016f3df16bf None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:13.296098 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc8c76f-f9ba-4c69-bf3b-a016f3df16bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:13.296935 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc8c76f-f9ba-4c69-bf3b-a016f3df16bf None admin] RBAC: Adding query filter params (role.id=70de044d6962417593b6d03571e7502e, scope.project.id=55a756ccaef14cb49b26b0d68dd0b466, user.id=04b5b06b0015450a95a28bbcc757cc8a) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:13.297243 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc8c76f-f9ba-4c69-bf3b-a016f3df16bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:13.298772 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc8c76f-f9ba-4c69-bf3b-a016f3df16bf None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:13.303849 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb48c9fd-8c67-4e8a-bb66-afe6834e3d2d None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:13.309891 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 263/534] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 15 19:15:13 2022] GET /identity/v3/role_assignments?user.id=04b5b06b0015450a95a28bbcc757cc8a&scope.project.id=55a756ccaef14cb49b26b0d68dd0b466&role.id=70de044d6962417593b6d03571e7502e => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:13.309891 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc8c76f-f9ba-4c69-bf3b-a016f3df16bf None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:13.392808 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328c4fe4-e8a1-438d-9dda-27dac8b7a898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:13.393639 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328c4fe4-e8a1-438d-9dda-27dac8b7a898 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:13.394015 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328c4fe4-e8a1-438d-9dda-27dac8b7a898 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:13.394369 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328c4fe4-e8a1-438d-9dda-27dac8b7a898 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:13.415665 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328c4fe4-e8a1-438d-9dda-27dac8b7a898 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:13.417236 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328c4fe4-e8a1-438d-9dda-27dac8b7a898 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:13.445860 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-328c4fe4-e8a1-438d-9dda-27dac8b7a898 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:13.445860 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:13.445860 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:13.445860 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:13.445860 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:13.445860 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:13.445860 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:13.445860 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:13.445860 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:13.445860 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:13.445860 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:13.445860 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:13.445860 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:15:13.445860 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:15:13.445860 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:15:13.446776 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:15:13.446776 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:13.446776 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:13.446776 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:13.446776 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:13.446776 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:13.446776 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:13.446776 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:13.446776 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:13.446776 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:13.446776 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:13.446776 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:13.446776 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:13.446776 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:13.446776 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:13.446776 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:13.446776 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:13.447708 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:15:13.447708 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:15:13.447708 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:13.447708 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:13.453968 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 272/535] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:15:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:13.455404 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328c4fe4-e8a1-438d-9dda-27dac8b7a898 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:13.467667 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:13.493701 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc8c76f-f9ba-4c69-bf3b-a016f3df16bf None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:13.557732 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c8bc71-e34c-44a4-9db9-1712a9d72211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:13.558754 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c8bc71-e34c-44a4-9db9-1712a9d72211 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:13.559409 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c8bc71-e34c-44a4-9db9-1712a9d72211 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:13.559614 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c8bc71-e34c-44a4-9db9-1712a9d72211 None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:13.560510 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c8bc71-e34c-44a4-9db9-1712a9d72211 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:13.560923 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c8bc71-e34c-44a4-9db9-1712a9d72211 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:13.562619 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c8bc71-e34c-44a4-9db9-1712a9d72211 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:13.565912 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328c4fe4-e8a1-438d-9dda-27dac8b7a898 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:13.578560 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 264/536] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jul 15 19:15:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:13.591115 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c8bc71-e34c-44a4-9db9-1712a9d72211 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:13.606678 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef74d275-7504-4808-a1ac-183d1f5fbe33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:13.608933 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef74d275-7504-4808-a1ac-183d1f5fbe33 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:13.609661 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef74d275-7504-4808-a1ac-183d1f5fbe33 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:13.610386 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef74d275-7504-4808-a1ac-183d1f5fbe33 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:13.612006 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef74d275-7504-4808-a1ac-183d1f5fbe33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:13.612771 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef74d275-7504-4808-a1ac-183d1f5fbe33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:13.615720 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef74d275-7504-4808-a1ac-183d1f5fbe33 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:13.624390 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:13.635942 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 273/537] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:15:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:13.642155 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef74d275-7504-4808-a1ac-183d1f5fbe33 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:13.738185 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:13.763840 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c8bc71-e34c-44a4-9db9-1712a9d72211 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:13.799947 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56705956-f114-4d2d-8003-bada3d4885cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:13.800605 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56705956-f114-4d2d-8003-bada3d4885cc None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:13.800882 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56705956-f114-4d2d-8003-bada3d4885cc None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:13.801152 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56705956-f114-4d2d-8003-bada3d4885cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:13.802010 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56705956-f114-4d2d-8003-bada3d4885cc None admin] RBAC: Adding query filter params (role.id=70de044d6962417593b6d03571e7502e, scope.project.id=55a756ccaef14cb49b26b0d68dd0b466, user.id=c86c2df3d378490ca7ea497a73c19d40) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:13.802315 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56705956-f114-4d2d-8003-bada3d4885cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:13.805600 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56705956-f114-4d2d-8003-bada3d4885cc None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:13.810849 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 265/538] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 15 19:15:13 2022] GET /identity/v3/role_assignments?user.id=c86c2df3d378490ca7ea497a73c19d40&scope.project.id=55a756ccaef14cb49b26b0d68dd0b466&role.id=70de044d6962417593b6d03571e7502e => generated 259 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:13.875977 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef74d275-7504-4808-a1ac-183d1f5fbe33 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:14.007758 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4854ec90-60fa-4e3a-a12a-c1539ab0fbec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:14.008508 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4854ec90-60fa-4e3a-a12a-c1539ab0fbec None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:14.008795 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4854ec90-60fa-4e3a-a12a-c1539ab0fbec None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:14.009072 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4854ec90-60fa-4e3a-a12a-c1539ab0fbec None admin] PATH_INFO: `/v3/projects/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:14.029618 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4854ec90-60fa-4e3a-a12a-c1539ab0fbec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:14.031129 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4854ec90-60fa-4e3a-a12a-c1539ab0fbec None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:14.077638 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-4854ec90-60fa-4e3a-a12a-c1539ab0fbec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:15:14.077638 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:14.077638 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:14.077638 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:14.077638 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:14.077638 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:14.077638 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:14.077638 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:14.077638 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:14.077638 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:14.077638 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:14.077638 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:14.077638 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 19:15:14.077638 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 19:15:14.077638 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 19:15:14.078590 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 19:15:14.078590 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:14.078590 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:14.078590 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:14.078590 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:14.078590 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:14.078590 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:14.078590 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:14.078590 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:14.078590 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:14.078590 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:14.078590 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:14.078590 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:14.078590 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:14.078590 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:14.078590 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:14.078590 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:14.079920 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 19:15:14.079920 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 19:15:14.079920 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:15:14.079920 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:15:14.079920 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:15:14.079920 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:15:14.079920 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:15:14.079920 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:14.079920 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 274/539] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 15 19:15:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 437 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:15:14.088160 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56705956-f114-4d2d-8003-bada3d4885cc None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:14.195376 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa4327-2516-4f24-8f7e-3eecc9d289b3 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:14.195693 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa4327-2516-4f24-8f7e-3eecc9d289b3 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:14.195943 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa4327-2516-4f24-8f7e-3eecc9d289b3 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:14.197497 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 275/540] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:15:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:15:14.203172 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2494e10-9ebe-4abb-af6b-f78d947e4e76 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:14.203172 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2494e10-9ebe-4abb-af6b-f78d947e4e76 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:14.203535 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2494e10-9ebe-4abb-af6b-f78d947e4e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:14.231892 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-c2494e10-9ebe-4abb-af6b-f78d947e4e76 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:15:14.233459 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-c2494e10-9ebe-4abb-af6b-f78d947e4e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:15:14.248026 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:14.292153 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56705956-f114-4d2d-8003-bada3d4885cc None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:14.361866 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 276/541] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:15:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:15:14.367021 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d70c3f4-7fce-4ac8-9a1c-dad18d9c3544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:14.367978 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d70c3f4-7fce-4ac8-9a1c-dad18d9c3544 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:14.368477 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d70c3f4-7fce-4ac8-9a1c-dad18d9c3544 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:14.368953 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d70c3f4-7fce-4ac8-9a1c-dad18d9c3544 None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:14.369985 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d70c3f4-7fce-4ac8-9a1c-dad18d9c3544 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:14.370516 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d70c3f4-7fce-4ac8-9a1c-dad18d9c3544 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:14.372184 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d70c3f4-7fce-4ac8-9a1c-dad18d9c3544 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:14.394292 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 266/542] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jul 15 19:15:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:14.399070 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2494e10-9ebe-4abb-af6b-f78d947e4e76 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:14.425802 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d70c3f4-7fce-4ac8-9a1c-dad18d9c3544 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:14.506999 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:14.590926 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2494e10-9ebe-4abb-af6b-f78d947e4e76 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:14.613322 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d70c3f4-7fce-4ac8-9a1c-dad18d9c3544 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:14.648495 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a4beac-90bd-40d2-86e1-d5a479a98ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:14.649165 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a4beac-90bd-40d2-86e1-d5a479a98ee7 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:14.649426 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a4beac-90bd-40d2-86e1-d5a479a98ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:14.649691 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a4beac-90bd-40d2-86e1-d5a479a98ee7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:14.650562 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a4beac-90bd-40d2-86e1-d5a479a98ee7 None admin] RBAC: Adding query filter params (role.id=70de044d6962417593b6d03571e7502e, scope.project.id=55a756ccaef14cb49b26b0d68dd0b466, user.id=b2a283d7d3594e37943bda4529d0a775) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:14.650867 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a4beac-90bd-40d2-86e1-d5a479a98ee7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:14.653371 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a4beac-90bd-40d2-86e1-d5a479a98ee7 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:14.667293 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 277/543] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 15 19:15:14 2022] GET /identity/v3/role_assignments?user.id=b2a283d7d3594e37943bda4529d0a775&scope.project.id=55a756ccaef14cb49b26b0d68dd0b466&role.id=70de044d6962417593b6d03571e7502e => generated 259 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:14.669746 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac118b95-b133-499d-a29f-b50029e34314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:14.670619 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac118b95-b133-499d-a29f-b50029e34314 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:14.670894 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac118b95-b133-499d-a29f-b50029e34314 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:14.671166 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac118b95-b133-499d-a29f-b50029e34314 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:14.674627 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:14.686361 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac118b95-b133-499d-a29f-b50029e34314 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:15:14.694937 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-ac118b95-b133-499d-a29f-b50029e34314 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:15:14.694937 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:14.694937 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:14.694937 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:14.694937 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:14.694937 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:14.694937 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:14.694937 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:14.694937 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:14.694937 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:14.694937 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:14.694937 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:14.694937 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:15:14.694937 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:15:14.694937 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:15:14.694937 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:15:14.695827 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:15:14.695827 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:15:14.695827 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:15:14.695827 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:15:14.695827 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:14.695827 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:14.695827 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:14.695827 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:14.695827 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:14.695827 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:14.695827 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:14.695827 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:14.695827 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:14.695827 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:14.695827 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:14.695827 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:14.696783 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:14.696783 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:14.696783 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:14.696783 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:14.696783 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:15:14.696783 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:15:14.696783 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:15:14.696783 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:15:14.696783 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:15:14.696783 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:15:14.696783 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:15:14.696783 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:14.696783 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 267/544] 10.222.0.33 () {56 vars in 1102 bytes} [Fri Jul 15 19:15:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:14.703866 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a4beac-90bd-40d2-86e1-d5a479a98ee7 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:14.853390 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a4beac-90bd-40d2-86e1-d5a479a98ee7 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:14.878911 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:14.890749 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275c62dd-9a78-4872-ab01-82fe2e04af93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:14.891424 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275c62dd-9a78-4872-ab01-82fe2e04af93 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:14.891704 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275c62dd-9a78-4872-ab01-82fe2e04af93 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:14.895726 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275c62dd-9a78-4872-ab01-82fe2e04af93 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:14.895726 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275c62dd-9a78-4872-ab01-82fe2e04af93 None admin] RBAC: Adding query filter params (name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:14.895726 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275c62dd-9a78-4872-ab01-82fe2e04af93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:14.895726 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275c62dd-9a78-4872-ab01-82fe2e04af93 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:14.905095 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 278/545] 10.222.0.33 () {56 vars in 1103 bytes} [Fri Jul 15 19:15:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:14.910608 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac118b95-b133-499d-a29f-b50029e34314 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:15.039012 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac118b95-b133-499d-a29f-b50029e34314 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:15.084897 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2785efc1-d02f-442c-bc41-8ff8eac23b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:15.086007 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2785efc1-d02f-442c-bc41-8ff8eac23b4b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:15.086487 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2785efc1-d02f-442c-bc41-8ff8eac23b4b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:15.086927 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2785efc1-d02f-442c-bc41-8ff8eac23b4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:15.105562 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2785efc1-d02f-442c-bc41-8ff8eac23b4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:15.107297 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2785efc1-d02f-442c-bc41-8ff8eac23b4b None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:15.129569 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-2785efc1-d02f-442c-bc41-8ff8eac23b4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:15.129569 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:15.129569 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:15.129569 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:15.129569 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:15.129569 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:15.129569 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:15.129569 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:15.129569 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:15.129569 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:15.129569 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:15.129569 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:15.129569 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:15:15.129569 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:15:15.129569 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:15:15.130653 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:15:15.130653 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:15.130653 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:15.130653 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:15.130653 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:15.130653 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:15.130653 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:15.130653 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:15.130653 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:15.130653 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:15.130653 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:15.130653 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:15.130653 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:15.130653 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:15.130653 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:15.130653 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:15.130653 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:15.131796 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:15:15.131796 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:15:15.131796 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:15.131796 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:15.133656 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 268/546] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:15:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:15.141791 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275c62dd-9a78-4872-ab01-82fe2e04af93 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:15.275802 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275c62dd-9a78-4872-ab01-82fe2e04af93 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:15.323962 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b66113-fa7f-4214-8483-b5938f7231c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:15.324637 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b66113-fa7f-4214-8483-b5938f7231c7 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:15.324920 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b66113-fa7f-4214-8483-b5938f7231c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:15.325198 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b66113-fa7f-4214-8483-b5938f7231c7 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:15.325976 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b66113-fa7f-4214-8483-b5938f7231c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:15.326289 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b66113-fa7f-4214-8483-b5938f7231c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:15.327772 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b66113-fa7f-4214-8483-b5938f7231c7 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:15.345943 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 279/547] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:15:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:15.353235 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2785efc1-d02f-442c-bc41-8ff8eac23b4b None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:15.436034 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:15.508474 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2785efc1-d02f-442c-bc41-8ff8eac23b4b None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:15.556928 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6711557f-22fc-421e-b9cc-e6cf775c1682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:15.557733 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6711557f-22fc-421e-b9cc-e6cf775c1682 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:15.558044 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6711557f-22fc-421e-b9cc-e6cf775c1682 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:15.558325 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6711557f-22fc-421e-b9cc-e6cf775c1682 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:15.565598 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6711557f-22fc-421e-b9cc-e6cf775c1682 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:15.567083 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6711557f-22fc-421e-b9cc-e6cf775c1682 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:15.586582 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-6711557f-22fc-421e-b9cc-e6cf775c1682 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 15 19:15:15.586582 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:15.586582 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:15.586582 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:15.586582 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:15.586582 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:15.586582 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:15.586582 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:15.586582 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:15.586582 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:15.586582 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:15.586582 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:15.586582 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 19:15:15.586582 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 19:15:15.586582 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 19:15:15.586582 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 19:15:15.587729 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:15.587729 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:15.587729 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:15:15.587729 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:15:15.587729 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:15:15.587729 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:15:15.587729 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:15.587729 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:15.587729 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:15.587729 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:15.587729 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:15.587729 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:15.587729 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:15.587729 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:15.587729 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:15.587729 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:15.587729 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:15.587729 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:15.588655 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:15.588655 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:15.588655 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 19:15:15.588655 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 19:15:15.588655 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 19:15:15.588655 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 19:15:15.588655 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 19:15:15.588655 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 19:15:15.588655 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 15 19:15:15.588655 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:15.592414 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 269/548] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jul 15 19:15:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:15.597988 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b66113-fa7f-4214-8483-b5938f7231c7 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:15.666443 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:15.729488 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b66113-fa7f-4214-8483-b5938f7231c7 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:15.765695 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab2ad5-5635-4191-bb77-eec084952ade None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:15.766036 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab2ad5-5635-4191-bb77-eec084952ade None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:15.766265 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab2ad5-5635-4191-bb77-eec084952ade None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:15.767769 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 270/549] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:15:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 19:15:15.781995 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8742bf4-2e1e-4d90-bfe7-4bb01d214f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:15.782770 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8742bf4-2e1e-4d90-bfe7-4bb01d214f75 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:15.783059 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8742bf4-2e1e-4d90-bfe7-4bb01d214f75 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:15.783395 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8742bf4-2e1e-4d90-bfe7-4bb01d214f75 None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:15.784261 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8742bf4-2e1e-4d90-bfe7-4bb01d214f75 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:15.784546 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8742bf4-2e1e-4d90-bfe7-4bb01d214f75 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:15.786043 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8742bf4-2e1e-4d90-bfe7-4bb01d214f75 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:15.791633 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30ebcbf-0ed8-41c2-ae7e-de4339ea4576 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:15.791924 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30ebcbf-0ed8-41c2-ae7e-de4339ea4576 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:15.792403 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30ebcbf-0ed8-41c2-ae7e-de4339ea4576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:15.821300 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 280/550] 10.222.0.33 () {56 vars in 1135 bytes} [Fri Jul 15 19:15:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:15.823432 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09af9801-55f3-4985-ab95-d713f10f2ba0 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:15.823799 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09af9801-55f3-4985-ab95-d713f10f2ba0 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:15.824217 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09af9801-55f3-4985-ab95-d713f10f2ba0 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:15.826420 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 281/551] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:15:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:15:15.827352 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09af9801-55f3-4985-ab95-d713f10f2ba0 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:15.829302 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-b30ebcbf-0ed8-41c2-ae7e-de4339ea4576 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:15:15.831271 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b30ebcbf-0ed8-41c2-ae7e-de4339ea4576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:15:15.945788 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 271/552] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:15:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:15:15.945788 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0166e976-b1b4-46f0-a7c1-de7380d8f68b None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:15.945788 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0166e976-b1b4-46f0-a7c1-de7380d8f68b None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:15.945788 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0166e976-b1b4-46f0-a7c1-de7380d8f68b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:15.957385 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-0166e976-b1b4-46f0-a7c1-de7380d8f68b None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:15:15.959736 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0166e976-b1b4-46f0-a7c1-de7380d8f68b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:15:15.963589 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09af9801-55f3-4985-ab95-d713f10f2ba0 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:15.965589 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:15.999612 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:16.000874 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e29f749-d8fd-4ca8-85ae-007fad4b501f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:16.001848 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e29f749-d8fd-4ca8-85ae-007fad4b501f None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:16.002367 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e29f749-d8fd-4ca8-85ae-007fad4b501f None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:16.002848 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e29f749-d8fd-4ca8-85ae-007fad4b501f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:16.053803 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 272/553] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:15:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:15:16.053803 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0166e976-b1b4-46f0-a7c1-de7380d8f68b None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:16.099885 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e29f749-d8fd-4ca8-85ae-007fad4b501f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:16.101390 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e29f749-d8fd-4ca8-85ae-007fad4b501f None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:16.183291 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0166e976-b1b4-46f0-a7c1-de7380d8f68b None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:16.220169 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53e5c48-c586-4883-85cb-150d71968095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:16.221036 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53e5c48-c586-4883-85cb-150d71968095 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:16.221244 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53e5c48-c586-4883-85cb-150d71968095 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:16.221637 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53e5c48-c586-4883-85cb-150d71968095 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:16.221862 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 282/554] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 15 19:15:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 19:15:16.222883 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e29f749-d8fd-4ca8-85ae-007fad4b501f None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:16.238145 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:16.238441 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53e5c48-c586-4883-85cb-150d71968095 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:15:16.247014 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-a53e5c48-c586-4883-85cb-150d71968095 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:15:16.247014 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:16.247014 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:16.247014 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:16.247014 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:16.247014 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:16.247014 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:16.247014 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:16.247014 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:16.247014 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:16.247014 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:16.247014 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:16.247014 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:15:16.247014 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:15:16.247014 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:15:16.247014 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:15:16.248073 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:15:16.248073 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:15:16.248073 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:15:16.248073 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:15:16.248073 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:16.248073 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:16.248073 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:16.248073 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:16.248073 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:16.248073 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:16.248073 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:16.248073 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:16.248073 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:16.248073 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:16.248073 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:16.248073 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:16.248897 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:16.248897 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:16.248897 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:16.248897 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:16.248897 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:15:16.248897 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:15:16.248897 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:15:16.248897 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:15:16.248897 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:15:16.248897 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:15:16.248897 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:15:16.248897 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:16.248897 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 273/555] 10.222.0.33 () {56 vars in 1102 bytes} [Fri Jul 15 19:15:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:16.256817 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53e5c48-c586-4883-85cb-150d71968095 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:16.289279 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:16.337296 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e29f749-d8fd-4ca8-85ae-007fad4b501f None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:16.377182 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53e5c48-c586-4883-85cb-150d71968095 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:16.382906 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3180e40e-1adc-436b-be51-b0dcfa44c6af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:16.383889 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180e40e-1adc-436b-be51-b0dcfa44c6af None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:16.384318 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180e40e-1adc-436b-be51-b0dcfa44c6af None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:16.384757 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180e40e-1adc-436b-be51-b0dcfa44c6af None admin] PATH_INFO: `/v3/roles/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:16.397423 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3180e40e-1adc-436b-be51-b0dcfa44c6af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:15:16.405036 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-3180e40e-1adc-436b-be51-b0dcfa44c6af None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:15:16.405036 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:16.405036 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:16.405036 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:16.405036 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:16.405036 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:16.405036 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:16.405036 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:16.405036 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:16.405036 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:16.405036 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:16.405036 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:16.405036 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:15:16.405036 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:15:16.405036 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:15:16.405036 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:15:16.405992 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:15:16.405992 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:15:16.405992 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:15:16.405992 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:15:16.405992 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:16.405992 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:16.405992 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:16.405992 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:16.405992 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:16.405992 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:16.405992 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:16.405992 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:16.405992 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:16.405992 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:16.405992 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:16.405992 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:16.406885 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:16.406885 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:16.406885 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:16.406885 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:16.406885 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:15:16.406885 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:15:16.406885 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:15:16.406885 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:15:16.406885 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:15:16.406885 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:15:16.406885 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:15:16.406885 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:16.406885 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 283/556] 10.222.0.33 () {56 vars in 1102 bytes} [Fri Jul 15 19:15:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:16.407551 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3180e40e-1adc-436b-be51-b0dcfa44c6af None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:16.421282 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd54795-7dfa-4159-ac2b-94737631ecbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:16.422128 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd54795-7dfa-4159-ac2b-94737631ecbe None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:16.422409 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd54795-7dfa-4159-ac2b-94737631ecbe None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:16.422680 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd54795-7dfa-4159-ac2b-94737631ecbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:16.434745 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd54795-7dfa-4159-ac2b-94737631ecbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:16.436191 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd54795-7dfa-4159-ac2b-94737631ecbe None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:16.454810 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-ddd54795-7dfa-4159-ac2b-94737631ecbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:16.454810 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:16.454810 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:16.454810 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:16.454810 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:16.454810 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:16.454810 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:16.454810 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:16.454810 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:16.454810 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:16.454810 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:16.454810 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:16.454810 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:15:16.454810 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:15:16.454810 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:15:16.455674 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:15:16.455674 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:16.455674 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:16.455674 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:16.455674 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:16.455674 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:16.455674 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:16.455674 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:16.455674 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:16.455674 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:16.455674 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:16.455674 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:16.455674 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:16.455674 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:16.455674 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:16.455674 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:16.455674 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:16.457042 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:15:16.457042 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:15:16.457042 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:16.457042 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:16.460817 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 274/557] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:15:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:16.462196 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aec7d0b-e244-4390-8953-5088695e83ad None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:16.462486 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aec7d0b-e244-4390-8953-5088695e83ad None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:16.462768 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aec7d0b-e244-4390-8953-5088695e83ad None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:16.464063 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 275/558] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:15:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:15:16.464751 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aec7d0b-e244-4390-8953-5088695e83ad None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:16.479859 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:16.519745 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3180e40e-1adc-436b-be51-b0dcfa44c6af None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:16.549448 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:16.557838 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbeeb27-551f-413f-9adc-6a8fd16b0b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:16.558513 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbeeb27-551f-413f-9adc-6a8fd16b0b42 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:16.559152 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbeeb27-551f-413f-9adc-6a8fd16b0b42 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:16.559152 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbeeb27-551f-413f-9adc-6a8fd16b0b42 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:16.559818 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbeeb27-551f-413f-9adc-6a8fd16b0b42 None admin] RBAC: Adding query filter params (name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:16.560114 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbeeb27-551f-413f-9adc-6a8fd16b0b42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:16.561484 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbeeb27-551f-413f-9adc-6a8fd16b0b42 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:16.570785 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 284/559] 10.222.0.33 () {56 vars in 1103 bytes} [Fri Jul 15 19:15:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:16.571573 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbeeb27-551f-413f-9adc-6a8fd16b0b42 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:16.576865 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aec7d0b-e244-4390-8953-5088695e83ad None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:16.617592 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084c0cee-5892-4414-acc8-de6541dbcdca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:16.618295 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084c0cee-5892-4414-acc8-de6541dbcdca None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:16.618580 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084c0cee-5892-4414-acc8-de6541dbcdca None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:16.618964 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084c0cee-5892-4414-acc8-de6541dbcdca None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:16.619662 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084c0cee-5892-4414-acc8-de6541dbcdca None admin] RBAC: Adding query filter params (name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:16.620914 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084c0cee-5892-4414-acc8-de6541dbcdca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:16.622498 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084c0cee-5892-4414-acc8-de6541dbcdca None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:16.633328 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 276/560] 10.222.0.33 () {56 vars in 1103 bytes} [Fri Jul 15 19:15:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:16.634453 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc880b05-c809-4840-8ee5-f4b678bdacfe None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:16.634744 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc880b05-c809-4840-8ee5-f4b678bdacfe None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:16.635024 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc880b05-c809-4840-8ee5-f4b678bdacfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:16.652025 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-cc880b05-c809-4840-8ee5-f4b678bdacfe None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:15:16.653221 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-cc880b05-c809-4840-8ee5-f4b678bdacfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:15:16.683112 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbeeb27-551f-413f-9adc-6a8fd16b0b42 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:16.708679 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:16.719543 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4fb66c-5f84-4237-aee3-a4d3185c11d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:16.720429 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4fb66c-5f84-4237-aee3-a4d3185c11d4 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:16.720715 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4fb66c-5f84-4237-aee3-a4d3185c11d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:16.721051 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4fb66c-5f84-4237-aee3-a4d3185c11d4 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:16.721714 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4fb66c-5f84-4237-aee3-a4d3185c11d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:16.722051 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4fb66c-5f84-4237-aee3-a4d3185c11d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:16.725813 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 277/561] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:15:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:15:16.725955 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4fb66c-5f84-4237-aee3-a4d3185c11d4 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:16.726076 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc880b05-c809-4840-8ee5-f4b678bdacfe None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:16.747085 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 285/562] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:15:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:16.748438 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4fb66c-5f84-4237-aee3-a4d3185c11d4 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:16.821702 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:16.852385 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc880b05-c809-4840-8ee5-f4b678bdacfe None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:16.862162 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4fb66c-5f84-4237-aee3-a4d3185c11d4 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:16.891122 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3b09fb-25f2-4d35-9c08-d7d27d95cc3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:16.891911 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3b09fb-25f2-4d35-9c08-d7d27d95cc3c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:16.892212 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3b09fb-25f2-4d35-9c08-d7d27d95cc3c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:16.892454 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3b09fb-25f2-4d35-9c08-d7d27d95cc3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:16.897911 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34269dbe-a976-4ba0-97c1-81e8630f964c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:16.898706 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34269dbe-a976-4ba0-97c1-81e8630f964c None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:16.898987 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34269dbe-a976-4ba0-97c1-81e8630f964c None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:16.899256 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34269dbe-a976-4ba0-97c1-81e8630f964c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:16.903334 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3b09fb-25f2-4d35-9c08-d7d27d95cc3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:16.904780 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3b09fb-25f2-4d35-9c08-d7d27d95cc3c None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:16.909993 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34269dbe-a976-4ba0-97c1-81e8630f964c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:16.911427 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34269dbe-a976-4ba0-97c1-81e8630f964c None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:16.922400 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-ea3b09fb-25f2-4d35-9c08-d7d27d95cc3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:16.922400 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:16.922400 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:16.922400 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:16.922400 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:16.922400 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:16.922400 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:16.922400 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:16.922400 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:16.922400 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:16.922400 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:16.922400 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:16.922400 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:15:16.922400 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:15:16.922400 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:15:16.923319 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:15:16.923319 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:16.923319 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:16.923319 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:16.923319 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:16.923319 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:16.923319 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:16.923319 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:16.923319 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:16.923319 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:16.923319 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:16.923319 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:16.923319 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:16.923319 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:16.923319 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:16.923319 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:16.923319 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:16.929744 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:15:16.929744 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:15:16.929744 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:16.929744 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:16.929744 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 278/563] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:15:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:16.929744 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3b09fb-25f2-4d35-9c08-d7d27d95cc3c None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:16.930285 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-34269dbe-a976-4ba0-97c1-81e8630f964c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:16.930285 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:16.930285 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:16.930285 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:16.930285 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:16.930285 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:16.930285 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:16.930285 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:16.930285 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:16.930285 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:16.930285 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:16.930285 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:16.930285 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:15:16.930285 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:15:16.930285 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:15:16.931103 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:15:16.931103 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:16.931103 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:16.931103 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:16.931103 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:16.931103 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:16.931103 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:16.931103 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:16.931103 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:16.931103 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:16.931103 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:16.931103 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:16.931103 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:16.931103 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:16.931103 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:16.931103 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:16.931103 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:16.932013 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:15:16.932013 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:15:16.932013 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:16.932013 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:16.935761 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 286/564] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:15:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:16.936632 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34269dbe-a976-4ba0-97c1-81e8630f964c None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:16.972174 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:17.041539 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34269dbe-a976-4ba0-97c1-81e8630f964c None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:17.042958 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3b09fb-25f2-4d35-9c08-d7d27d95cc3c None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:17.068883 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:17.079855 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d9624a-cc48-4646-bec3-75e7499286dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:17.080767 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d9624a-cc48-4646-bec3-75e7499286dd None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:17.081105 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d9624a-cc48-4646-bec3-75e7499286dd None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:17.083285 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d9624a-cc48-4646-bec3-75e7499286dd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:17.083401 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c44a8e-4baf-4186-a434-4a5e2614562d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:17.083863 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c44a8e-4baf-4186-a434-4a5e2614562d None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:17.084141 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c44a8e-4baf-4186-a434-4a5e2614562d None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:17.084412 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c44a8e-4baf-4186-a434-4a5e2614562d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:17.091462 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c44a8e-4baf-4186-a434-4a5e2614562d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:17.092462 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d9624a-cc48-4646-bec3-75e7499286dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:15:17.092896 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c44a8e-4baf-4186-a434-4a5e2614562d None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:17.101080 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-00d9624a-cc48-4646-bec3-75e7499286dd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:15:17.101080 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:17.101080 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:17.101080 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:17.101080 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:17.101080 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:17.101080 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:17.101080 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:17.101080 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:17.101080 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:17.101080 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:17.101080 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:17.101080 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:15:17.101080 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:15:17.101080 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:15:17.101080 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:15:17.102243 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:15:17.102243 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:15:17.102243 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:15:17.102243 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:15:17.102243 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:17.102243 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:17.102243 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:17.102243 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:17.102243 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:17.102243 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:17.102243 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:17.102243 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:17.102243 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:17.102243 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:17.102243 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:17.102243 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:17.103290 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:17.103290 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:17.103290 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:17.103290 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:17.103290 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:15:17.103290 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:15:17.103290 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:15:17.103290 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:15:17.103290 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:15:17.103290 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:15:17.103290 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:15:17.103290 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:17.110269 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 279/565] 10.222.0.33 () {56 vars in 1102 bytes} [Fri Jul 15 19:15:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:17.110269 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d9624a-cc48-4646-bec3-75e7499286dd None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:17.110460 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-14c44a8e-4baf-4186-a434-4a5e2614562d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:15:17.110460 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:17.110460 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:17.110460 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:17.110460 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:17.110460 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:17.110460 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:17.110460 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:17.110460 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:17.110460 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:17.110460 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:17.110460 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:17.110460 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 19:15:17.110460 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 19:15:17.110460 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 19:15:17.111271 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 19:15:17.111271 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:17.111271 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:17.111271 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:17.111271 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:17.111271 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:17.111271 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:17.111271 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:17.111271 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:17.111271 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:17.111271 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:17.111271 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:17.111271 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:17.111271 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:17.111271 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:17.111271 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:17.111271 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:17.112172 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 19:15:17.112172 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 19:15:17.112172 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:15:17.112172 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:15:17.112172 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:15:17.112172 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:15:17.112172 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:15:17.112172 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:17.112172 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 287/566] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 15 19:15:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:17.112172 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c44a8e-4baf-4186-a434-4a5e2614562d None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:17.214976 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:17.217859 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d9624a-cc48-4646-bec3-75e7499286dd None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:17.221043 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c44a8e-4baf-4186-a434-4a5e2614562d None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:17.251613 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398f3c06-d4ba-4238-b288-cdddc1808f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:17.252263 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398f3c06-d4ba-4238-b288-cdddc1808f1b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:17.252542 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398f3c06-d4ba-4238-b288-cdddc1808f1b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:17.252813 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398f3c06-d4ba-4238-b288-cdddc1808f1b None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:17.253525 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398f3c06-d4ba-4238-b288-cdddc1808f1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:17.253859 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398f3c06-d4ba-4238-b288-cdddc1808f1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:17.254884 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e4da0d-247f-4d20-8e8b-32f29b7e4e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:17.255234 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398f3c06-d4ba-4238-b288-cdddc1808f1b None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:17.255544 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e4da0d-247f-4d20-8e8b-32f29b7e4e11 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:17.255855 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e4da0d-247f-4d20-8e8b-32f29b7e4e11 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:17.256136 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e4da0d-247f-4d20-8e8b-32f29b7e4e11 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:17.257035 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e4da0d-247f-4d20-8e8b-32f29b7e4e11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:17.257784 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e4da0d-247f-4d20-8e8b-32f29b7e4e11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:17.259166 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e4da0d-247f-4d20-8e8b-32f29b7e4e11 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:17.266976 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 280/567] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:15:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:17.269257 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398f3c06-d4ba-4238-b288-cdddc1808f1b None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:17.275880 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 288/568] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:15:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:17.275880 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e4da0d-247f-4d20-8e8b-32f29b7e4e11 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:17.315117 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:17.369697 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398f3c06-d4ba-4238-b288-cdddc1808f1b None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:17.371968 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e4da0d-247f-4d20-8e8b-32f29b7e4e11 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:17.415869 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54bf736-ac86-4b98-b5a3-e44e67094de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:17.416055 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4108865-5848-44fb-a3ed-fcde5f337829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:17.416524 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54bf736-ac86-4b98-b5a3-e44e67094de8 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:17.416636 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4108865-5848-44fb-a3ed-fcde5f337829 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:17.416811 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54bf736-ac86-4b98-b5a3-e44e67094de8 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:17.416904 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4108865-5848-44fb-a3ed-fcde5f337829 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:17.417091 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54bf736-ac86-4b98-b5a3-e44e67094de8 None admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:17.417183 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4108865-5848-44fb-a3ed-fcde5f337829 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:17.417859 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4108865-5848-44fb-a3ed-fcde5f337829 None admin] RBAC: Adding query filter params (name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:17.417953 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54bf736-ac86-4b98-b5a3-e44e67094de8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:17.418161 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4108865-5848-44fb-a3ed-fcde5f337829 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:17.418255 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54bf736-ac86-4b98-b5a3-e44e67094de8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:17.419529 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4108865-5848-44fb-a3ed-fcde5f337829 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:17.419632 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54bf736-ac86-4b98-b5a3-e44e67094de8 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:17.427465 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 281/569] 10.222.0.33 () {56 vars in 1103 bytes} [Fri Jul 15 19:15:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:17.428652 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10153a01-de32-40e5-809f-e64c7e98fdee None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:17.428941 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10153a01-de32-40e5-809f-e64c7e98fdee None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:17.429215 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10153a01-de32-40e5-809f-e64c7e98fdee None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:17.430599 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 282/570] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:15:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:15:17.431194 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10153a01-de32-40e5-809f-e64c7e98fdee None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:17.437603 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 289/571] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jul 15 19:15:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:17.437603 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54bf736-ac86-4b98-b5a3-e44e67094de8 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:17.453994 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:17.533973 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54bf736-ac86-4b98-b5a3-e44e67094de8 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:17.537848 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10153a01-de32-40e5-809f-e64c7e98fdee None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:17.544535 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:17.567440 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9002aa14-28c3-48a2-b613-fcc8cf711118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:17.568243 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9002aa14-28c3-48a2-b613-fcc8cf711118 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:17.568521 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9002aa14-28c3-48a2-b613-fcc8cf711118 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:17.568793 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9002aa14-28c3-48a2-b613-fcc8cf711118 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:17.571628 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7034068c-2757-47d3-991f-51e912e96918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:17.572414 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7034068c-2757-47d3-991f-51e912e96918 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:17.572700 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7034068c-2757-47d3-991f-51e912e96918 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:17.572979 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7034068c-2757-47d3-991f-51e912e96918 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:17.577102 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9002aa14-28c3-48a2-b613-fcc8cf711118 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:17.578558 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9002aa14-28c3-48a2-b613-fcc8cf711118 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:17.581141 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7034068c-2757-47d3-991f-51e912e96918 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:17.582601 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7034068c-2757-47d3-991f-51e912e96918 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:17.593239 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-9002aa14-28c3-48a2-b613-fcc8cf711118 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 15 19:15:17.593239 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:17.593239 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:17.593239 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:17.593239 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:17.593239 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:17.593239 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:17.593239 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:17.593239 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:17.593239 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:17.593239 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:17.593239 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:17.593239 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 19:15:17.593239 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 19:15:17.593239 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 19:15:17.593239 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 19:15:17.594439 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:17.594439 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:17.594439 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:15:17.594439 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:15:17.594439 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:15:17.594439 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:15:17.594439 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:17.594439 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:17.594439 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:17.594439 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:17.594439 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:17.594439 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:17.594439 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:17.594439 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:17.594439 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:17.594439 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:17.594439 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:17.594439 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:17.595659 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:17.595659 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:17.595659 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 19:15:17.595659 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 19:15:17.595659 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 19:15:17.595659 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 19:15:17.595659 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 19:15:17.595659 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 19:15:17.595659 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 15 19:15:17.595659 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:17.595659 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 290/572] 10.222.0.33 () {56 vars in 1093 bytes} [Fri Jul 15 19:15:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:17.595659 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14dffe3-70e2-48fe-8b03-d379843ca519 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:17.596417 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14dffe3-70e2-48fe-8b03-d379843ca519 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:17.596417 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14dffe3-70e2-48fe-8b03-d379843ca519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:17.598045 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-7034068c-2757-47d3-991f-51e912e96918 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 15 19:15:17.598045 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:17.598045 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:17.598045 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:17.598045 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:17.598045 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:17.598045 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:17.598045 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:17.598045 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:17.598045 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:17.598045 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:17.598045 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:17.598045 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 19:15:17.598045 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 19:15:17.598045 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 19:15:17.598045 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 19:15:17.599219 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:17.599219 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:17.599219 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:15:17.599219 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:15:17.599219 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:15:17.599219 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:15:17.599219 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:17.599219 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:17.599219 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:17.599219 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:17.599219 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:17.599219 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:17.599219 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:17.599219 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:17.599219 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:17.599219 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:17.599219 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:17.599219 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:17.600410 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:17.600410 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:17.600410 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 19:15:17.600410 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 19:15:17.600410 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 19:15:17.600410 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 19:15:17.600410 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 19:15:17.600410 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 19:15:17.600410 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 15 19:15:17.600410 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:17.602671 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 283/573] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 15 19:15:17 2022] GET /identity/v3/users/glance => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:17.603550 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7034068c-2757-47d3-991f-51e912e96918 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:17.613723 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-a14dffe3-70e2-48fe-8b03-d379843ca519 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:15:17.615469 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-a14dffe3-70e2-48fe-8b03-d379843ca519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:15:17.693343 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 291/574] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:15:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:15:17.694247 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14dffe3-70e2-48fe-8b03-d379843ca519 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:17.701954 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:17.714020 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7034068c-2757-47d3-991f-51e912e96918 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:17.749656 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f984f6fb-307e-443f-9bf5-f47e7199f548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:17.750485 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f984f6fb-307e-443f-9bf5-f47e7199f548 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:17.750786 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f984f6fb-307e-443f-9bf5-f47e7199f548 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:17.751062 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f984f6fb-307e-443f-9bf5-f47e7199f548 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:17.761552 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f984f6fb-307e-443f-9bf5-f47e7199f548 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:17.763223 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f984f6fb-307e-443f-9bf5-f47e7199f548 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:17.782206 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-f984f6fb-307e-443f-9bf5-f47e7199f548 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:17.782206 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:17.782206 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:17.782206 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:17.782206 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:17.782206 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:17.782206 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:17.782206 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:17.782206 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:17.782206 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:17.782206 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:17.782206 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:17.782206 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:15:17.782206 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:15:17.782206 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:15:17.783183 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:15:17.783183 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:17.783183 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:17.783183 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:17.783183 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:17.783183 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:17.783183 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:17.783183 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:17.783183 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:17.783183 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:17.783183 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:17.783183 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:17.783183 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:17.783183 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:17.783183 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:17.783183 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:17.783183 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:17.784086 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:15:17.784086 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:15:17.784086 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:17.784086 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:17.784086 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 284/575] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:15:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:17.785008 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f984f6fb-307e-443f-9bf5-f47e7199f548 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:17.797611 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14dffe3-70e2-48fe-8b03-d379843ca519 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:17.802868 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:17.831763 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7c6ff6-4312-48e0-8df4-e852faee1aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:17.832413 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c6ff6-4312-48e0-8df4-e852faee1aca None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:17.832693 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c6ff6-4312-48e0-8df4-e852faee1aca None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:17.832967 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c6ff6-4312-48e0-8df4-e852faee1aca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:17.833885 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7c6ff6-4312-48e0-8df4-e852faee1aca None admin] RBAC: Adding query filter params (role.id=70de044d6962417593b6d03571e7502e, scope.project.id=55a756ccaef14cb49b26b0d68dd0b466, user.id=ef76f7fe39a3499aaeebf85c176804f3) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:17.834259 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7c6ff6-4312-48e0-8df4-e852faee1aca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:17.835632 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7c6ff6-4312-48e0-8df4-e852faee1aca None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:17.840976 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 292/576] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 15 19:15:17 2022] GET /identity/v3/role_assignments?user.id=ef76f7fe39a3499aaeebf85c176804f3&scope.project.id=55a756ccaef14cb49b26b0d68dd0b466&role.id=70de044d6962417593b6d03571e7502e => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:17.841745 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7c6ff6-4312-48e0-8df4-e852faee1aca None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:17.894604 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f984f6fb-307e-443f-9bf5-f47e7199f548 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:17.919107 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:17.935805 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c6f2b9-1c22-4c62-a1db-faa914693425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:17.936600 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c6f2b9-1c22-4c62-a1db-faa914693425 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:17.936877 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c6f2b9-1c22-4c62-a1db-faa914693425 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:17.937145 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c6f2b9-1c22-4c62-a1db-faa914693425 None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:17.938023 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c6f2b9-1c22-4c62-a1db-faa914693425 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:17.938329 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c6f2b9-1c22-4c62-a1db-faa914693425 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:17.939817 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c6f2b9-1c22-4c62-a1db-faa914693425 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:17.956542 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7c6ff6-4312-48e0-8df4-e852faee1aca None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:17.959236 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 285/577] 10.222.0.33 () {56 vars in 1133 bytes} [Fri Jul 15 19:15:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:17.960116 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c6f2b9-1c22-4c62-a1db-faa914693425 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:18.002887 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f824bf7f-3827-419c-9189-95bfe7fca98b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:18.003574 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f824bf7f-3827-419c-9189-95bfe7fca98b None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:18.003860 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f824bf7f-3827-419c-9189-95bfe7fca98b None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:18.004140 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f824bf7f-3827-419c-9189-95bfe7fca98b None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:18.004986 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f824bf7f-3827-419c-9189-95bfe7fca98b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:18.005288 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f824bf7f-3827-419c-9189-95bfe7fca98b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:18.006797 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f824bf7f-3827-419c-9189-95bfe7fca98b None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:18.030402 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 293/578] 10.222.0.33 () {56 vars in 1137 bytes} [Fri Jul 15 19:15:17 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:18.030402 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f824bf7f-3827-419c-9189-95bfe7fca98b None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:18.049377 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:18.073414 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c6f2b9-1c22-4c62-a1db-faa914693425 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:18.107909 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df11306e-04e2-412b-bfa2-b69a3fd1ecd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:18.108705 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df11306e-04e2-412b-bfa2-b69a3fd1ecd8 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:18.108987 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df11306e-04e2-412b-bfa2-b69a3fd1ecd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:18.109262 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df11306e-04e2-412b-bfa2-b69a3fd1ecd8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:18.120192 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df11306e-04e2-412b-bfa2-b69a3fd1ecd8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:15:18.127945 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-df11306e-04e2-412b-bfa2-b69a3fd1ecd8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:15:18.127945 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:18.127945 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:18.127945 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:18.127945 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:18.127945 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:18.127945 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:18.127945 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:18.127945 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:18.127945 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:18.127945 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:18.127945 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:18.127945 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:15:18.127945 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:15:18.127945 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:15:18.127945 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:15:18.128879 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:15:18.128879 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:15:18.128879 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:15:18.128879 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:15:18.128879 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:18.128879 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:18.128879 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:18.128879 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:18.128879 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:18.128879 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:18.128879 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:18.128879 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:18.128879 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:18.128879 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:18.128879 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:18.128879 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:18.136097 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:18.136097 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:18.136097 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:18.136097 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:18.136097 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:15:18.136097 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:15:18.136097 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:15:18.136097 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:15:18.136097 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:15:18.136097 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:15:18.136097 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:15:18.136097 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:18.136097 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 286/579] 10.222.0.33 () {56 vars in 1102 bytes} [Fri Jul 15 19:15:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:18.136097 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df11306e-04e2-412b-bfa2-b69a3fd1ecd8 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:18.149593 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f824bf7f-3827-419c-9189-95bfe7fca98b None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:18.177211 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:18.186597 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95025c5d-b72c-48ce-a055-79f7007ccf8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:18.187296 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95025c5d-b72c-48ce-a055-79f7007ccf8d None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:18.189712 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95025c5d-b72c-48ce-a055-79f7007ccf8d None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:18.189712 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95025c5d-b72c-48ce-a055-79f7007ccf8d None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:18.189712 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95025c5d-b72c-48ce-a055-79f7007ccf8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:18.189712 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95025c5d-b72c-48ce-a055-79f7007ccf8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:18.190284 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95025c5d-b72c-48ce-a055-79f7007ccf8d None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:18.209258 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 294/580] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:15:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:18.209258 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95025c5d-b72c-48ce-a055-79f7007ccf8d None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:18.258963 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df11306e-04e2-412b-bfa2-b69a3fd1ecd8 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:18.295062 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42a889f-f123-47c9-913a-4e41401b4b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:18.295860 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a889f-f123-47c9-913a-4e41401b4b48 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:18.296137 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a889f-f123-47c9-913a-4e41401b4b48 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:18.296410 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a889f-f123-47c9-913a-4e41401b4b48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:18.299782 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:18.308690 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42a889f-f123-47c9-913a-4e41401b4b48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:18.312553 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42a889f-f123-47c9-913a-4e41401b4b48 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:18.323214 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95025c5d-b72c-48ce-a055-79f7007ccf8d None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:18.327713 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-f42a889f-f123-47c9-913a-4e41401b4b48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:18.327713 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:18.327713 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:18.327713 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:18.327713 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:18.327713 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:18.327713 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:18.327713 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:18.327713 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:18.327713 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:18.327713 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:18.327713 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:18.327713 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:15:18.327713 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:15:18.327713 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:15:18.328911 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:15:18.328911 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:18.328911 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:18.328911 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:18.328911 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:18.328911 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:18.328911 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:18.328911 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:18.328911 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:18.328911 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:18.328911 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:18.328911 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:18.328911 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:18.328911 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:18.328911 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:18.328911 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:18.328911 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:18.333084 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:15:18.333084 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:15:18.333084 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:18.333084 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:18.333649 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 287/581] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:15:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:18.334502 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42a889f-f123-47c9-913a-4e41401b4b48 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:18.359683 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6e84d4-6e6a-44a6-8cd5-7b46b9ad9a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:18.361800 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e84d4-6e6a-44a6-8cd5-7b46b9ad9a0e None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:18.361800 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e84d4-6e6a-44a6-8cd5-7b46b9ad9a0e None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:18.361800 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e84d4-6e6a-44a6-8cd5-7b46b9ad9a0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:18.374075 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6e84d4-6e6a-44a6-8cd5-7b46b9ad9a0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:18.375507 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6e84d4-6e6a-44a6-8cd5-7b46b9ad9a0e None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:18.393040 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-dd6e84d4-6e6a-44a6-8cd5-7b46b9ad9a0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:18.393040 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:18.393040 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:18.393040 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:18.393040 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:18.393040 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:18.393040 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:18.393040 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:18.393040 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:18.393040 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:18.393040 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:18.393040 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:18.393040 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:15:18.393040 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:15:18.393040 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:15:18.394001 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:15:18.394001 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:18.394001 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:18.394001 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:18.394001 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:18.394001 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:18.394001 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:18.394001 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:18.394001 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:18.394001 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:18.394001 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:18.394001 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:18.394001 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:18.394001 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:18.394001 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:18.394001 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:18.394001 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:18.394960 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:15:18.394960 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:15:18.394960 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:18.394960 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:18.394960 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 295/582] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:15:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:18.394960 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6e84d4-6e6a-44a6-8cd5-7b46b9ad9a0e None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:18.441127 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:18.463543 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42a889f-f123-47c9-913a-4e41401b4b48 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:18.501824 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83989baa-6eb0-4fe8-8884-023edd5c239a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:18.502500 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83989baa-6eb0-4fe8-8884-023edd5c239a None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:18.502784 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83989baa-6eb0-4fe8-8884-023edd5c239a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:18.503061 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83989baa-6eb0-4fe8-8884-023edd5c239a None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:18.503812 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83989baa-6eb0-4fe8-8884-023edd5c239a None admin] RBAC: Adding query filter params (name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:18.504119 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83989baa-6eb0-4fe8-8884-023edd5c239a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:18.504580 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6e84d4-6e6a-44a6-8cd5-7b46b9ad9a0e None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:18.506359 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83989baa-6eb0-4fe8-8884-023edd5c239a None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:18.514742 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 288/583] 10.222.0.33 () {56 vars in 1103 bytes} [Fri Jul 15 19:15:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:18.515563 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83989baa-6eb0-4fe8-8884-023edd5c239a None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:18.546709 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db415f49-2239-4933-a76e-3f7e4c9bbfba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:18.547866 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db415f49-2239-4933-a76e-3f7e4c9bbfba None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:18.548381 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db415f49-2239-4933-a76e-3f7e4c9bbfba None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:18.548757 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db415f49-2239-4933-a76e-3f7e4c9bbfba None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:18.556745 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db415f49-2239-4933-a76e-3f7e4c9bbfba None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:18.558248 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db415f49-2239-4933-a76e-3f7e4c9bbfba None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:18.569602 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:18.573515 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-db415f49-2239-4933-a76e-3f7e4c9bbfba None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 15 19:15:18.573515 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:18.573515 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:18.573515 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:18.573515 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:18.573515 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:18.573515 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:18.573515 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:18.573515 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:18.573515 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:18.573515 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:18.573515 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:18.573515 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 19:15:18.573515 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 19:15:18.573515 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 19:15:18.573515 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 19:15:18.574492 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:18.574492 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:18.574492 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:15:18.574492 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:15:18.574492 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:15:18.574492 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:15:18.574492 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:18.574492 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:18.574492 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:18.574492 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:18.574492 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:18.574492 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:18.574492 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:18.574492 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:18.574492 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:18.574492 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:18.574492 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:18.574492 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:18.575436 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:18.575436 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:18.575436 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 19:15:18.575436 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 19:15:18.575436 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 19:15:18.575436 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 19:15:18.575436 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 19:15:18.575436 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 19:15:18.575436 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 15 19:15:18.575436 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:18.575436 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 296/584] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 15 19:15:18 2022] GET /identity/v3/users/cinder => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:18.576043 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db415f49-2239-4933-a76e-3f7e4c9bbfba None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:18.627157 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83989baa-6eb0-4fe8-8884-023edd5c239a None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:18.663179 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751438a6-44a1-4881-b290-2c94f4ca3b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:18.663846 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751438a6-44a1-4881-b290-2c94f4ca3b67 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:18.664125 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751438a6-44a1-4881-b290-2c94f4ca3b67 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:18.664402 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751438a6-44a1-4881-b290-2c94f4ca3b67 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:18.665123 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751438a6-44a1-4881-b290-2c94f4ca3b67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:18.665428 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751438a6-44a1-4881-b290-2c94f4ca3b67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:18.666843 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751438a6-44a1-4881-b290-2c94f4ca3b67 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:18.683723 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 289/585] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:15:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:18.683723 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751438a6-44a1-4881-b290-2c94f4ca3b67 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:18.693532 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:18.696720 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db415f49-2239-4933-a76e-3f7e4c9bbfba None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:18.731000 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f795848-bd62-4dda-a7c5-20061ed7f231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:18.731576 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f795848-bd62-4dda-a7c5-20061ed7f231 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:18.731861 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f795848-bd62-4dda-a7c5-20061ed7f231 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:18.732136 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f795848-bd62-4dda-a7c5-20061ed7f231 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:18.732863 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f795848-bd62-4dda-a7c5-20061ed7f231 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:18.733171 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f795848-bd62-4dda-a7c5-20061ed7f231 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:18.734668 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f795848-bd62-4dda-a7c5-20061ed7f231 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:18.747841 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 297/586] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:15:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:18.748659 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f795848-bd62-4dda-a7c5-20061ed7f231 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:18.795158 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751438a6-44a1-4881-b290-2c94f4ca3b67 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:18.807116 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:18.832615 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d810f8-449c-4252-8a94-e77319238fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:18.833411 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d810f8-449c-4252-8a94-e77319238fc6 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:18.834345 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d810f8-449c-4252-8a94-e77319238fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:18.834641 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d810f8-449c-4252-8a94-e77319238fc6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:18.845910 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d810f8-449c-4252-8a94-e77319238fc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:18.847378 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d810f8-449c-4252-8a94-e77319238fc6 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:18.860181 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f795848-bd62-4dda-a7c5-20061ed7f231 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:18.865970 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-34d810f8-449c-4252-8a94-e77319238fc6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:18.865970 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:18.865970 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:18.865970 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:18.865970 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:18.865970 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:18.865970 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:18.865970 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:18.865970 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:18.865970 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:18.865970 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:18.865970 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:18.865970 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:15:18.865970 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:15:18.865970 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:15:18.866855 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:15:18.866855 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:18.866855 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:18.866855 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:18.866855 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:18.866855 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:18.866855 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:18.866855 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:18.866855 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:18.866855 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:18.866855 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:18.866855 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:18.866855 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:18.866855 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:18.866855 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:18.866855 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:18.866855 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:18.867764 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:15:18.867764 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:15:18.867764 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:18.867764 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:18.872134 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 290/587] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:15:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:18.872487 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d810f8-449c-4252-8a94-e77319238fc6 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:18.899699 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8d76a6-52ca-469b-a788-297c34b58ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:18.900459 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8d76a6-52ca-469b-a788-297c34b58ee9 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:18.900651 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8d76a6-52ca-469b-a788-297c34b58ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:18.900924 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8d76a6-52ca-469b-a788-297c34b58ee9 None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:18.901797 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8d76a6-52ca-469b-a788-297c34b58ee9 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:18.902124 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8d76a6-52ca-469b-a788-297c34b58ee9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:18.903563 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8d76a6-52ca-469b-a788-297c34b58ee9 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:18.926359 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 298/588] 10.222.0.33 () {56 vars in 1137 bytes} [Fri Jul 15 19:15:18 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:18.926359 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8d76a6-52ca-469b-a788-297c34b58ee9 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:18.944437 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:18.988651 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d810f8-449c-4252-8a94-e77319238fc6 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:19.025513 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca281dd-bfc9-41bd-a345-f64af3b54ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:19.026447 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca281dd-bfc9-41bd-a345-f64af3b54ca6 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:19.026745 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca281dd-bfc9-41bd-a345-f64af3b54ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:19.027009 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca281dd-bfc9-41bd-a345-f64af3b54ca6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:19.028808 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8d76a6-52ca-469b-a788-297c34b58ee9 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:19.033969 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca281dd-bfc9-41bd-a345-f64af3b54ca6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:19.035404 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca281dd-bfc9-41bd-a345-f64af3b54ca6 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:19.050322 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-8ca281dd-bfc9-41bd-a345-f64af3b54ca6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:15:19.050322 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:19.050322 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:19.050322 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:19.050322 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:19.050322 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:19.050322 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:19.050322 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:19.050322 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:19.050322 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:19.050322 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:19.050322 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:19.050322 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 19:15:19.050322 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 19:15:19.050322 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 19:15:19.051765 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 19:15:19.051765 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:19.051765 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:19.051765 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:19.051765 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:19.051765 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:19.051765 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:19.051765 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:19.051765 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:19.051765 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:19.051765 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:19.051765 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:19.051765 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:19.051765 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:19.051765 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:19.051765 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:19.051765 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:19.053218 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 19:15:19.053218 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 19:15:19.053218 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:15:19.053218 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:15:19.053218 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:15:19.053218 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:15:19.053218 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:15:19.053218 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:19.053218 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 291/589] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 15 19:15:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:19.053218 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca281dd-bfc9-41bd-a345-f64af3b54ca6 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:19.068591 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5a9164-7149-42f8-b925-8b73e2e1c791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:19.069394 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a9164-7149-42f8-b925-8b73e2e1c791 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:19.069678 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a9164-7149-42f8-b925-8b73e2e1c791 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:19.069989 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a9164-7149-42f8-b925-8b73e2e1c791 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:19.077046 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5a9164-7149-42f8-b925-8b73e2e1c791 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:19.078027 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:19.078518 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5a9164-7149-42f8-b925-8b73e2e1c791 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:19.092816 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-4e5a9164-7149-42f8-b925-8b73e2e1c791 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:15:19.092816 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:19.092816 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:19.092816 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:19.092816 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:19.092816 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:19.092816 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:19.092816 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:19.092816 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:19.092816 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:19.092816 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:19.092816 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:19.092816 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 19:15:19.092816 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 19:15:19.092816 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 19:15:19.093919 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 19:15:19.093919 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:19.093919 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:19.093919 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:19.093919 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:19.093919 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:19.093919 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:19.093919 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:19.093919 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:19.093919 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:19.093919 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:19.093919 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:19.093919 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:19.093919 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:19.093919 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:19.093919 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:19.093919 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:19.095255 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 19:15:19.095255 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 19:15:19.095255 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:15:19.095255 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:15:19.095255 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:15:19.095255 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:15:19.095255 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:15:19.095255 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:19.097952 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 299/590] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 15 19:15:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:19.099900 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5a9164-7149-42f8-b925-8b73e2e1c791 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:19.161280 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca281dd-bfc9-41bd-a345-f64af3b54ca6 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:19.185020 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:19.198402 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789c556b-35bf-4573-9d2e-fab56a5129f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:19.204189 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789c556b-35bf-4573-9d2e-fab56a5129f0 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:19.204189 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789c556b-35bf-4573-9d2e-fab56a5129f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:19.204189 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789c556b-35bf-4573-9d2e-fab56a5129f0 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:19.204189 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789c556b-35bf-4573-9d2e-fab56a5129f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:19.204189 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789c556b-35bf-4573-9d2e-fab56a5129f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:19.204189 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789c556b-35bf-4573-9d2e-fab56a5129f0 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:19.208170 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5a9164-7149-42f8-b925-8b73e2e1c791 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:19.218876 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 292/591] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:15:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:19.218876 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789c556b-35bf-4573-9d2e-fab56a5129f0 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:19.251405 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866a2f4e-5f78-4a00-92c4-ab14761f6bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:19.253012 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866a2f4e-5f78-4a00-92c4-ab14761f6bd9 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:19.253296 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866a2f4e-5f78-4a00-92c4-ab14761f6bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:19.253574 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866a2f4e-5f78-4a00-92c4-ab14761f6bd9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:19.265829 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866a2f4e-5f78-4a00-92c4-ab14761f6bd9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:19.267302 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866a2f4e-5f78-4a00-92c4-ab14761f6bd9 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:19.285251 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-866a2f4e-5f78-4a00-92c4-ab14761f6bd9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:19.285251 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:19.285251 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:19.285251 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:19.285251 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:19.285251 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:19.285251 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:19.285251 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:19.285251 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:19.285251 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:19.285251 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:19.285251 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:19.285251 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:15:19.285251 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:15:19.285251 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:15:19.286411 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:15:19.286411 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:19.286411 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:19.286411 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:19.286411 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:19.286411 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:19.286411 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:19.286411 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:19.286411 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:19.286411 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:19.286411 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:19.286411 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:19.286411 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:19.286411 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:19.286411 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:19.286411 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:19.286411 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:19.292755 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:15:19.292755 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:15:19.292755 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:19.292755 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:19.292755 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 300/592] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:15:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:19.292755 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866a2f4e-5f78-4a00-92c4-ab14761f6bd9 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:19.311251 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:19.330672 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789c556b-35bf-4573-9d2e-fab56a5129f0 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:19.368762 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88676b5-ad88-42dd-8669-1d82e80b4716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:19.369430 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88676b5-ad88-42dd-8669-1d82e80b4716 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:19.369714 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88676b5-ad88-42dd-8669-1d82e80b4716 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:19.370021 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88676b5-ad88-42dd-8669-1d82e80b4716 None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:19.370812 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88676b5-ad88-42dd-8669-1d82e80b4716 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:19.371122 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88676b5-ad88-42dd-8669-1d82e80b4716 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:19.372571 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88676b5-ad88-42dd-8669-1d82e80b4716 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:19.385727 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 293/593] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jul 15 19:15:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:19.387029 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88676b5-ad88-42dd-8669-1d82e80b4716 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:19.399096 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866a2f4e-5f78-4a00-92c4-ab14761f6bd9 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:19.430374 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:19.436421 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44adc9f1-d240-4024-9c13-48f28a1e2c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:19.437222 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44adc9f1-d240-4024-9c13-48f28a1e2c1e None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:19.437608 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44adc9f1-d240-4024-9c13-48f28a1e2c1e None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:19.438024 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44adc9f1-d240-4024-9c13-48f28a1e2c1e None admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:19.440068 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44adc9f1-d240-4024-9c13-48f28a1e2c1e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:19.440068 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44adc9f1-d240-4024-9c13-48f28a1e2c1e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:19.440869 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44adc9f1-d240-4024-9c13-48f28a1e2c1e None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:19.455972 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 301/594] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jul 15 19:15:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:19.456976 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44adc9f1-d240-4024-9c13-48f28a1e2c1e None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:19.503233 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88676b5-ad88-42dd-8669-1d82e80b4716 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:19.538370 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cb897b-7b2b-423e-916f-0dfb7aa0d843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:19.539180 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cb897b-7b2b-423e-916f-0dfb7aa0d843 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:19.539462 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cb897b-7b2b-423e-916f-0dfb7aa0d843 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:19.539740 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cb897b-7b2b-423e-916f-0dfb7aa0d843 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:19.552453 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cb897b-7b2b-423e-916f-0dfb7aa0d843 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:19.553961 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cb897b-7b2b-423e-916f-0dfb7aa0d843 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:19.569147 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-02cb897b-7b2b-423e-916f-0dfb7aa0d843 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 15 19:15:19.569147 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:19.569147 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:19.569147 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:19.569147 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:19.569147 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:19.569147 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:19.569147 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:19.569147 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:19.569147 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:19.569147 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:19.569147 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:19.569147 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 19:15:19.569147 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 19:15:19.569147 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 19:15:19.569147 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 19:15:19.570192 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:19.570192 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:19.570192 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:15:19.570192 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:15:19.570192 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:15:19.570192 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:15:19.570192 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:19.570192 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:19.570192 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:19.570192 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:19.570192 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:19.570192 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:19.570192 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:19.570192 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:19.570192 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:19.570192 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:19.570192 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:19.570192 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:19.571172 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:19.571172 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:19.571172 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 19:15:19.571172 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 19:15:19.571172 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 19:15:19.571172 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 19:15:19.571172 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 19:15:19.571172 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 19:15:19.571172 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 15 19:15:19.571172 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:19.571172 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 294/595] 10.222.0.33 () {56 vars in 1102 bytes} [Fri Jul 15 19:15:19 2022] GET /identity/v3/users/neutron => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:19.571172 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cb897b-7b2b-423e-916f-0dfb7aa0d843 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:19.573230 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44adc9f1-d240-4024-9c13-48f28a1e2c1e None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:19.577328 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:19.613873 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea9adce-cecb-4a98-a60b-c6ca251e522c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:19.614692 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea9adce-cecb-4a98-a60b-c6ca251e522c None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:19.615608 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea9adce-cecb-4a98-a60b-c6ca251e522c None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:19.615982 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea9adce-cecb-4a98-a60b-c6ca251e522c None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:19.616823 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea9adce-cecb-4a98-a60b-c6ca251e522c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:19.617225 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea9adce-cecb-4a98-a60b-c6ca251e522c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:19.618624 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea9adce-cecb-4a98-a60b-c6ca251e522c None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:19.631166 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 302/596] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:15:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:19.632005 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea9adce-cecb-4a98-a60b-c6ca251e522c None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:19.689120 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:19.697712 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cb897b-7b2b-423e-916f-0dfb7aa0d843 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:19.734380 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c2d382-7557-482d-b967-73f380d362fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:19.735272 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c2d382-7557-482d-b967-73f380d362fa None admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:19.735571 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c2d382-7557-482d-b967-73f380d362fa None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:19.735825 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c2d382-7557-482d-b967-73f380d362fa None admin] PATH_INFO: `/v3/projects/55a756ccaef14cb49b26b0d68dd0b466/users/7673a3b5c0324e07a9535ac70ed7a491/roles/70de044d6962417593b6d03571e7502e` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:19.751531 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c2d382-7557-482d-b967-73f380d362fa None admin] RBAC: Authorizing `identity:create_grant(project_id=55a756ccaef14cb49b26b0d68dd0b466, user_id=7673a3b5c0324e07a9535ac70ed7a491, role_id=70de044d6962417593b6d03571e7502e)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:19.752970 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c2d382-7557-482d-b967-73f380d362fa None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:19.754743 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea9adce-cecb-4a98-a60b-c6ca251e522c None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:19.790604 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd8ac7d-2647-4069-8d9b-d66adeef5d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:19.791499 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd8ac7d-2647-4069-8d9b-d66adeef5d8f None admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:19.791781 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd8ac7d-2647-4069-8d9b-d66adeef5d8f None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:19.792054 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd8ac7d-2647-4069-8d9b-d66adeef5d8f None admin] PATH_INFO: `/v3/projects/55a756ccaef14cb49b26b0d68dd0b466/users/04b5b06b0015450a95a28bbcc757cc8a/roles/70de044d6962417593b6d03571e7502e` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:19.807429 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd8ac7d-2647-4069-8d9b-d66adeef5d8f None admin] RBAC: Authorizing `identity:create_grant(project_id=55a756ccaef14cb49b26b0d68dd0b466, user_id=04b5b06b0015450a95a28bbcc757cc8a, role_id=70de044d6962417593b6d03571e7502e)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:19.808934 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd8ac7d-2647-4069-8d9b-d66adeef5d8f None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:19.826229 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 295/597] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jul 15 19:15:19 2022] PUT /identity/v3/projects/55a756ccaef14cb49b26b0d68dd0b466/users/7673a3b5c0324e07a9535ac70ed7a491/roles/70de044d6962417593b6d03571e7502e => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:15:19.827084 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c2d382-7557-482d-b967-73f380d362fa None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:19.885951 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 303/598] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jul 15 19:15:19 2022] PUT /identity/v3/projects/55a756ccaef14cb49b26b0d68dd0b466/users/04b5b06b0015450a95a28bbcc757cc8a/roles/70de044d6962417593b6d03571e7502e => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:15:19.887285 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd8ac7d-2647-4069-8d9b-d66adeef5d8f None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:19.910351 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:19.984350 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c2d382-7557-482d-b967-73f380d362fa None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:20.021802 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:20.026713 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd8ac7d-2647-4069-8d9b-d66adeef5d8f None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:20.036240 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2b3386-2af6-4b6e-b042-3b5df9289927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:20.036240 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2b3386-2af6-4b6e-b042-3b5df9289927 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:20.036240 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2b3386-2af6-4b6e-b042-3b5df9289927 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:20.036240 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2b3386-2af6-4b6e-b042-3b5df9289927 None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:20.036607 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2b3386-2af6-4b6e-b042-3b5df9289927 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:20.036607 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2b3386-2af6-4b6e-b042-3b5df9289927 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:20.036607 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2b3386-2af6-4b6e-b042-3b5df9289927 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:20.059102 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 296/599] 10.222.0.33 () {56 vars in 1139 bytes} [Fri Jul 15 19:15:19 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:20.061533 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5360bf-e17a-44f3-b4b0-38061935115f None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:20.062088 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5360bf-e17a-44f3-b4b0-38061935115f None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:20.062570 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5360bf-e17a-44f3-b4b0-38061935115f None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:20.064873 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 297/600] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:15:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:15:20.065989 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5360bf-e17a-44f3-b4b0-38061935115f None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:20.085576 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213a009b-b663-4d91-9f6f-4d763615bd50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:20.086183 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a009b-b663-4d91-9f6f-4d763615bd50 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:20.086467 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a009b-b663-4d91-9f6f-4d763615bd50 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:20.086745 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a009b-b663-4d91-9f6f-4d763615bd50 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:20.096334 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213a009b-b663-4d91-9f6f-4d763615bd50 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:20.099479 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213a009b-b663-4d91-9f6f-4d763615bd50 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:20.114532 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-213a009b-b663-4d91-9f6f-4d763615bd50 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:15:20.114532 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:20.114532 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:20.114532 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:20.114532 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:20.114532 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:20.114532 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:20.114532 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:20.114532 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:20.114532 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:20.114532 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:20.114532 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:20.114532 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 19:15:20.114532 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 19:15:20.114532 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 19:15:20.115534 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 19:15:20.115534 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:20.115534 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:20.115534 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:20.115534 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:20.115534 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:20.115534 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:20.115534 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:20.115534 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:20.115534 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:20.115534 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:20.115534 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:20.115534 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:20.115534 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:20.115534 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:20.115534 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:20.115534 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:20.116446 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 19:15:20.116446 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 19:15:20.116446 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:15:20.116446 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:15:20.116446 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:15:20.116446 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:15:20.116446 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:15:20.116446 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:20.116446 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 304/601] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 15 19:15:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:20.117442 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d533bce8-302b-465a-af18-0247befe46aa None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:20.117613 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d533bce8-302b-465a-af18-0247befe46aa None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:20.119283 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d533bce8-302b-465a-af18-0247befe46aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:20.139407 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-d533bce8-302b-465a-af18-0247befe46aa None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:15:20.141216 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-d533bce8-302b-465a-af18-0247befe46aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:15:20.216913 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5360bf-e17a-44f3-b4b0-38061935115f None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:20.232657 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:20.232856 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 305/602] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:15:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:15:20.233942 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d533bce8-302b-465a-af18-0247befe46aa None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:20.263839 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f034bd-54af-4d60-aff0-15a8e3c42ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:20.265652 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f034bd-54af-4d60-aff0-15a8e3c42ab4 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:20.266309 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f034bd-54af-4d60-aff0-15a8e3c42ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:20.266893 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f034bd-54af-4d60-aff0-15a8e3c42ab4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:20.282802 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f034bd-54af-4d60-aff0-15a8e3c42ab4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:20.285204 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f034bd-54af-4d60-aff0-15a8e3c42ab4 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:20.311006 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-73f034bd-54af-4d60-aff0-15a8e3c42ab4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:20.311006 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:20.311006 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:20.311006 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:20.311006 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:20.311006 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:20.311006 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:20.311006 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:20.311006 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:20.311006 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:20.311006 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:20.311006 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:20.311006 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:15:20.311006 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:15:20.311006 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:15:20.311965 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:15:20.311965 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:20.311965 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:20.311965 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:20.311965 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:20.311965 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:20.311965 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:20.311965 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:20.311965 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:20.311965 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:20.311965 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:20.311965 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:20.311965 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:20.311965 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:20.311965 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:20.311965 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:20.311965 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:20.312874 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:15:20.312874 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:15:20.312874 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:20.312874 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:20.315746 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 298/603] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:15:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:20.317046 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f034bd-54af-4d60-aff0-15a8e3c42ab4 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:20.347449 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:20.360375 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d533bce8-302b-465a-af18-0247befe46aa None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:20.407239 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bac61a-2971-4af7-b9a3-d69809ee1053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:20.408057 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bac61a-2971-4af7-b9a3-d69809ee1053 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:20.408462 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bac61a-2971-4af7-b9a3-d69809ee1053 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:20.409096 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bac61a-2971-4af7-b9a3-d69809ee1053 None admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:20.410128 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bac61a-2971-4af7-b9a3-d69809ee1053 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:20.411198 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bac61a-2971-4af7-b9a3-d69809ee1053 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:20.413069 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bac61a-2971-4af7-b9a3-d69809ee1053 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:20.430268 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 306/604] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jul 15 19:15:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:20.431334 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bac61a-2971-4af7-b9a3-d69809ee1053 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:20.465282 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f034bd-54af-4d60-aff0-15a8e3c42ab4 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:20.506659 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75081476-f437-423f-b739-7fdda603cf86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:20.507721 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75081476-f437-423f-b739-7fdda603cf86 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:20.508153 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75081476-f437-423f-b739-7fdda603cf86 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:20.508713 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75081476-f437-423f-b739-7fdda603cf86 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:20.521706 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75081476-f437-423f-b739-7fdda603cf86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:15:20.530444 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-75081476-f437-423f-b739-7fdda603cf86 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:15:20.530444 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:20.530444 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:20.530444 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:20.530444 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:20.530444 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:20.530444 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:20.530444 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:20.530444 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:20.530444 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:20.530444 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:20.530444 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:20.530444 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:15:20.530444 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:15:20.530444 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:15:20.530444 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:15:20.531616 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:15:20.531616 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:15:20.531616 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:15:20.531616 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:15:20.531616 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:20.531616 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:20.531616 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:20.531616 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:20.531616 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:20.531616 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:20.531616 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:20.531616 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:20.531616 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:20.531616 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:20.531616 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:20.531616 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:20.532672 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:20.532672 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:20.532672 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:20.532672 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:20.532672 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:15:20.532672 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:15:20.532672 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:15:20.532672 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:15:20.532672 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:15:20.532672 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:15:20.532672 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:15:20.532672 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:20.536532 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 299/605] 10.222.0.33 () {56 vars in 1102 bytes} [Fri Jul 15 19:15:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:20.537526 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75081476-f437-423f-b739-7fdda603cf86 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:20.566530 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:20.567434 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bac61a-2971-4af7-b9a3-d69809ee1053 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:20.616925 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503eab43-d753-4acf-8372-e04a723524d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:20.617686 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503eab43-d753-4acf-8372-e04a723524d8 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:20.618102 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503eab43-d753-4acf-8372-e04a723524d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:20.618472 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503eab43-d753-4acf-8372-e04a723524d8 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:20.619607 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503eab43-d753-4acf-8372-e04a723524d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:20.620016 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503eab43-d753-4acf-8372-e04a723524d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:20.622716 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503eab43-d753-4acf-8372-e04a723524d8 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:20.640983 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 307/606] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:15:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:20.642313 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503eab43-d753-4acf-8372-e04a723524d8 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:20.655843 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:20.661818 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75081476-f437-423f-b739-7fdda603cf86 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:20.705452 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b75ebce-5699-4aa5-959e-8a9342ae1700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:20.706693 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b75ebce-5699-4aa5-959e-8a9342ae1700 None admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:20.707124 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b75ebce-5699-4aa5-959e-8a9342ae1700 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:20.707573 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b75ebce-5699-4aa5-959e-8a9342ae1700 None admin] PATH_INFO: `/v3/projects/55a756ccaef14cb49b26b0d68dd0b466/users/c86c2df3d378490ca7ea497a73c19d40/roles/70de044d6962417593b6d03571e7502e` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:20.722834 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b75ebce-5699-4aa5-959e-8a9342ae1700 None admin] RBAC: Authorizing `identity:create_grant(project_id=55a756ccaef14cb49b26b0d68dd0b466, user_id=c86c2df3d378490ca7ea497a73c19d40, role_id=70de044d6962417593b6d03571e7502e)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:20.724940 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b75ebce-5699-4aa5-959e-8a9342ae1700 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:20.755281 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503eab43-d753-4acf-8372-e04a723524d8 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:20.796754 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5013c29-edff-4b7d-9114-9b19c4c208bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:20.797351 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5013c29-edff-4b7d-9114-9b19c4c208bc None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:20.797631 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5013c29-edff-4b7d-9114-9b19c4c208bc None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:20.797996 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5013c29-edff-4b7d-9114-9b19c4c208bc None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:20.798740 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5013c29-edff-4b7d-9114-9b19c4c208bc None admin] RBAC: Adding query filter params (name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:20.799044 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5013c29-edff-4b7d-9114-9b19c4c208bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:20.800403 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5013c29-edff-4b7d-9114-9b19c4c208bc None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:20.811986 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 308/607] 10.222.0.33 () {56 vars in 1103 bytes} [Fri Jul 15 19:15:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:20.812652 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5013c29-edff-4b7d-9114-9b19c4c208bc None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:20.829012 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 300/608] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jul 15 19:15:20 2022] PUT /identity/v3/projects/55a756ccaef14cb49b26b0d68dd0b466/users/c86c2df3d378490ca7ea497a73c19d40/roles/70de044d6962417593b6d03571e7502e => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:15:20.831983 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b75ebce-5699-4aa5-959e-8a9342ae1700 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:20.922716 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:20.947892 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:20.961007 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b75ebce-5699-4aa5-959e-8a9342ae1700 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:21.005062 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee73bcbe-1f19-4f8a-83d8-e0c900fecf5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:21.006665 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee73bcbe-1f19-4f8a-83d8-e0c900fecf5e None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:21.007160 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee73bcbe-1f19-4f8a-83d8-e0c900fecf5e None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:21.007670 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee73bcbe-1f19-4f8a-83d8-e0c900fecf5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:21.016727 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5013c29-edff-4b7d-9114-9b19c4c208bc None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:21.020666 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee73bcbe-1f19-4f8a-83d8-e0c900fecf5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:21.023948 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee73bcbe-1f19-4f8a-83d8-e0c900fecf5e None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:21.048143 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-ee73bcbe-1f19-4f8a-83d8-e0c900fecf5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:21.048143 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:21.048143 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:21.048143 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:21.048143 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:21.048143 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:21.048143 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:21.048143 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:21.048143 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:21.048143 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:21.048143 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:21.048143 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:21.048143 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:15:21.048143 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:15:21.048143 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:15:21.049087 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:15:21.049087 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:21.049087 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:21.049087 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:21.049087 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:21.049087 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:21.049087 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:21.049087 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:21.049087 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:21.049087 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:21.049087 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:21.049087 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:21.049087 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:21.049087 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:21.049087 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:21.049087 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:21.049087 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:21.050064 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:15:21.050064 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:15:21.050064 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:21.050064 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:21.053412 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 301/609] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:15:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:21.060893 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee73bcbe-1f19-4f8a-83d8-e0c900fecf5e None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:21.122492 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50ada3a-b876-4f72-be2b-203649e663a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:21.123312 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50ada3a-b876-4f72-be2b-203649e663a5 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:21.123597 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50ada3a-b876-4f72-be2b-203649e663a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:21.123867 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50ada3a-b876-4f72-be2b-203649e663a5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:21.140596 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50ada3a-b876-4f72-be2b-203649e663a5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:21.142514 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50ada3a-b876-4f72-be2b-203649e663a5 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:21.169791 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-e50ada3a-b876-4f72-be2b-203649e663a5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:15:21.169791 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:21.169791 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:21.169791 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:21.169791 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:21.169791 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:21.169791 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:21.169791 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:21.169791 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:21.169791 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:21.169791 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:21.169791 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:21.169791 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 19:15:21.169791 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 19:15:21.169791 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 19:15:21.170937 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 19:15:21.170937 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:21.170937 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:21.170937 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:21.170937 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:21.170937 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:21.170937 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:21.170937 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:21.170937 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:21.170937 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:21.170937 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:21.170937 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:21.170937 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:21.170937 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:21.170937 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:21.170937 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:21.170937 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:21.172046 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 19:15:21.172046 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 19:15:21.172046 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:15:21.172046 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:15:21.172046 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:15:21.172046 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:15:21.172046 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:15:21.172046 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:21.174332 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 309/610] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 15 19:15:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:21.178367 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee73bcbe-1f19-4f8a-83d8-e0c900fecf5e None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:21.180841 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50ada3a-b876-4f72-be2b-203649e663a5 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:21.206418 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:21.219919 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f134d0-dab4-4b8f-adcf-50dedac8a5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:21.220727 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f134d0-dab4-4b8f-adcf-50dedac8a5e6 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:21.221066 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f134d0-dab4-4b8f-adcf-50dedac8a5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:21.221346 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f134d0-dab4-4b8f-adcf-50dedac8a5e6 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:21.222114 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f134d0-dab4-4b8f-adcf-50dedac8a5e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:21.222422 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f134d0-dab4-4b8f-adcf-50dedac8a5e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:21.224413 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f134d0-dab4-4b8f-adcf-50dedac8a5e6 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:21.244335 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 302/611] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:15:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:15:21.247390 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f134d0-dab4-4b8f-adcf-50dedac8a5e6 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:21.335199 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50ada3a-b876-4f72-be2b-203649e663a5 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:21.367907 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:21.373239 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc3e506-e5f7-4af8-9227-9f5abe2d2902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:21.373930 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc3e506-e5f7-4af8-9227-9f5abe2d2902 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:21.374214 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc3e506-e5f7-4af8-9227-9f5abe2d2902 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:21.374485 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc3e506-e5f7-4af8-9227-9f5abe2d2902 None admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:21.375544 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc3e506-e5f7-4af8-9227-9f5abe2d2902 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:21.375859 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc3e506-e5f7-4af8-9227-9f5abe2d2902 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:21.377342 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc3e506-e5f7-4af8-9227-9f5abe2d2902 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:21.389922 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 310/612] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jul 15 19:15:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:15:21.395570 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc3e506-e5f7-4af8-9227-9f5abe2d2902 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:21.444269 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f134d0-dab4-4b8f-adcf-50dedac8a5e6 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:21.515690 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc3e506-e5f7-4af8-9227-9f5abe2d2902 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:21.529988 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de147260-6826-482e-a0e2-da6a5103683b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:21.530798 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de147260-6826-482e-a0e2-da6a5103683b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:21.531081 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de147260-6826-482e-a0e2-da6a5103683b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:21.531353 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de147260-6826-482e-a0e2-da6a5103683b None admin] PATH_INFO: `/v3/users/swift` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:21.554644 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de147260-6826-482e-a0e2-da6a5103683b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:21.556477 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de147260-6826-482e-a0e2-da6a5103683b None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:21.562763 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d169f2-7037-4c57-b9e6-0102f2107098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:21.563666 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d169f2-7037-4c57-b9e6-0102f2107098 None admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:21.563951 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d169f2-7037-4c57-b9e6-0102f2107098 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:21.564228 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d169f2-7037-4c57-b9e6-0102f2107098 None admin] PATH_INFO: `/v3/projects/55a756ccaef14cb49b26b0d68dd0b466/users/b2a283d7d3594e37943bda4529d0a775/roles/70de044d6962417593b6d03571e7502e` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:21.580520 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d169f2-7037-4c57-b9e6-0102f2107098 None admin] RBAC: Authorizing `identity:create_grant(project_id=55a756ccaef14cb49b26b0d68dd0b466, user_id=b2a283d7d3594e37943bda4529d0a775, role_id=70de044d6962417593b6d03571e7502e)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:21.589843 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d169f2-7037-4c57-b9e6-0102f2107098 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:21.591570 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-de147260-6826-482e-a0e2-da6a5103683b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 15 19:15:21.591570 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:21.591570 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:21.591570 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:21.591570 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:21.591570 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:21.591570 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:21.591570 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:21.591570 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:21.591570 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:21.591570 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:21.591570 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:21.591570 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 19:15:21.591570 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 19:15:21.591570 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 19:15:21.591570 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 19:15:21.592472 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:21.592472 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:21.592472 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:15:21.592472 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:15:21.592472 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:15:21.592472 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:15:21.592472 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:21.592472 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:21.592472 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:21.592472 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:21.592472 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:21.592472 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:21.592472 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:21.592472 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:21.592472 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:21.592472 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:21.592472 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:21.592472 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:21.593429 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:21.593429 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:21.593429 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 19:15:21.593429 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 19:15:21.593429 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 19:15:21.593429 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 19:15:21.593429 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 19:15:21.593429 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 19:15:21.593429 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 15 19:15:21.593429 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:21.593429 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 303/613] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jul 15 19:15:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:21.601059 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de147260-6826-482e-a0e2-da6a5103683b None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:21.652330 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:21.666491 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 311/614] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jul 15 19:15:21 2022] PUT /identity/v3/projects/55a756ccaef14cb49b26b0d68dd0b466/users/b2a283d7d3594e37943bda4529d0a775/roles/70de044d6962417593b6d03571e7502e => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:15:21.726569 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de147260-6826-482e-a0e2-da6a5103683b None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:21.774295 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f486b6a-5bdb-41a7-b59c-8c510ab0ec2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:21.775283 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f486b6a-5bdb-41a7-b59c-8c510ab0ec2e None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:21.775715 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f486b6a-5bdb-41a7-b59c-8c510ab0ec2e None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:21.776139 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f486b6a-5bdb-41a7-b59c-8c510ab0ec2e None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:21.777164 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f486b6a-5bdb-41a7-b59c-8c510ab0ec2e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:21.777620 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f486b6a-5bdb-41a7-b59c-8c510ab0ec2e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:21.779318 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f486b6a-5bdb-41a7-b59c-8c510ab0ec2e None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:21.800482 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 304/615] 10.222.0.33 () {56 vars in 1135 bytes} [Fri Jul 15 19:15:21 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:21.806207 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d169f2-7037-4c57-b9e6-0102f2107098 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:21.902053 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:21.965183 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d169f2-7037-4c57-b9e6-0102f2107098 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:22.001694 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d5f4b8-286c-4521-becf-251e418aa987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:22.002534 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d5f4b8-286c-4521-becf-251e418aa987 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:22.002818 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d5f4b8-286c-4521-becf-251e418aa987 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:22.003097 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d5f4b8-286c-4521-becf-251e418aa987 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:22.016852 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d5f4b8-286c-4521-becf-251e418aa987 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:22.018349 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d5f4b8-286c-4521-becf-251e418aa987 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:22.042609 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-86d5f4b8-286c-4521-becf-251e418aa987 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:22.042609 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:22.042609 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:22.042609 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:22.042609 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:22.042609 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:22.042609 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:22.042609 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:22.042609 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:22.042609 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:22.042609 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:22.042609 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:22.042609 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:15:22.042609 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:15:22.042609 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:15:22.043578 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:15:22.043578 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:22.043578 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:22.043578 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:22.043578 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:22.043578 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:22.043578 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:22.043578 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:22.043578 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:22.043578 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:22.043578 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:22.043578 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:22.043578 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:22.043578 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:22.043578 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:22.043578 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:22.043578 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:22.044473 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:15:22.044473 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:15:22.044473 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:22.044473 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:22.048159 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 312/616] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:15:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:22.049406 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f486b6a-5bdb-41a7-b59c-8c510ab0ec2e None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:22.199374 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f486b6a-5bdb-41a7-b59c-8c510ab0ec2e None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:22.202131 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:22.235400 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad1c162-2ae6-4050-ac97-88acaf30aeb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:22.236074 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad1c162-2ae6-4050-ac97-88acaf30aeb5 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:22.236360 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad1c162-2ae6-4050-ac97-88acaf30aeb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:22.236637 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad1c162-2ae6-4050-ac97-88acaf30aeb5 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:22.237376 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad1c162-2ae6-4050-ac97-88acaf30aeb5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:22.237702 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad1c162-2ae6-4050-ac97-88acaf30aeb5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:22.239091 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad1c162-2ae6-4050-ac97-88acaf30aeb5 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:22.253388 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 305/617] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:15:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:22.259208 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d5f4b8-286c-4521-becf-251e418aa987 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:22.308702 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2f727b-9b91-4bb8-a8d5-fcca9f91a41a None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:22.309125 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2f727b-9b91-4bb8-a8d5-fcca9f91a41a None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:22.309289 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2f727b-9b91-4bb8-a8d5-fcca9f91a41a None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:22.311883 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 306/618] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:15:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 19:15:22.316055 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ba3612-8fdd-4e51-b628-c99a09aab0a5 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:22.316342 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ba3612-8fdd-4e51-b628-c99a09aab0a5 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:22.322927 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ba3612-8fdd-4e51-b628-c99a09aab0a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:22.336471 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-70ba3612-8fdd-4e51-b628-c99a09aab0a5 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:15:22.338455 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-70ba3612-8fdd-4e51-b628-c99a09aab0a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:15:22.372273 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:22.379073 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d5f4b8-286c-4521-becf-251e418aa987 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:22.415775 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e24c45-0d2e-43e5-bd1b-73de4af916b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:22.416804 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e24c45-0d2e-43e5-bd1b-73de4af916b0 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:22.417182 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e24c45-0d2e-43e5-bd1b-73de4af916b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:22.417552 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e24c45-0d2e-43e5-bd1b-73de4af916b0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:22.426768 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e24c45-0d2e-43e5-bd1b-73de4af916b0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:22.428289 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e24c45-0d2e-43e5-bd1b-73de4af916b0 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:22.432427 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 307/619] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:15:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 19:15:22.433866 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f587f3c-dd06-4250-b6a5-87ade9a9f1da None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:22.434161 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f587f3c-dd06-4250-b6a5-87ade9a9f1da None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:22.434491 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f587f3c-dd06-4250-b6a5-87ade9a9f1da None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:22.436068 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 308/620] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:15:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:15:22.441142 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bac82e9-8a5c-4718-9479-57ec3ba1607f None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:22.441628 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bac82e9-8a5c-4718-9479-57ec3ba1607f None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:22.442104 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bac82e9-8a5c-4718-9479-57ec3ba1607f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:22.444608 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-98e24c45-0d2e-43e5-bd1b-73de4af916b0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:15:22.444608 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:22.444608 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:22.444608 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:22.444608 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:22.444608 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:22.444608 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:22.444608 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:22.444608 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:22.444608 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:22.444608 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:22.444608 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:22.444608 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 19:15:22.444608 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 19:15:22.444608 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 19:15:22.445525 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 19:15:22.445525 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:22.445525 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:22.445525 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:22.445525 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:22.445525 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:22.445525 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:22.445525 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:22.445525 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:22.445525 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:22.445525 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:22.445525 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:22.445525 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:22.445525 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:22.445525 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:22.445525 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:22.445525 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:22.446483 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 19:15:22.446483 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 19:15:22.446483 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:15:22.446483 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:15:22.446483 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:15:22.446483 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:15:22.446483 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:15:22.446483 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:22.446483 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 313/621] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 15 19:15:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:15:22.451740 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e24c45-0d2e-43e5-bd1b-73de4af916b0 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:22.475595 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-2bac82e9-8a5c-4718-9479-57ec3ba1607f None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:15:22.477352 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-2bac82e9-8a5c-4718-9479-57ec3ba1607f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:15:22.561162 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e24c45-0d2e-43e5-bd1b-73de4af916b0 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:22.576173 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 309/622] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:15:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 19:15:22.577353 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bac82e9-8a5c-4718-9479-57ec3ba1607f None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:22.604601 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:22.606349 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0c3beb-2d51-4faa-b56e-761eb91f890d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:22.607336 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0c3beb-2d51-4faa-b56e-761eb91f890d None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:22.607930 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0c3beb-2d51-4faa-b56e-761eb91f890d None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:22.608488 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0c3beb-2d51-4faa-b56e-761eb91f890d None admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:22.609603 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0c3beb-2d51-4faa-b56e-761eb91f890d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:22.610415 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0c3beb-2d51-4faa-b56e-761eb91f890d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:22.612411 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0c3beb-2d51-4faa-b56e-761eb91f890d None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:22.630690 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 314/623] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jul 15 19:15:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:15:22.631897 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0c3beb-2d51-4faa-b56e-761eb91f890d None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:22.652048 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:22.697304 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bac82e9-8a5c-4718-9479-57ec3ba1607f None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:22.734891 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2f45fd-b47f-4aa2-9b4c-b531aeee34c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:22.735612 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2f45fd-b47f-4aa2-9b4c-b531aeee34c7 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:22.735948 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2f45fd-b47f-4aa2-9b4c-b531aeee34c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:22.736219 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2f45fd-b47f-4aa2-9b4c-b531aeee34c7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:22.751404 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2f45fd-b47f-4aa2-9b4c-b531aeee34c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:15:22.758881 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0c3beb-2d51-4faa-b56e-761eb91f890d None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:22.762757 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-6f2f45fd-b47f-4aa2-9b4c-b531aeee34c7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:15:22.762757 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:22.762757 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:22.762757 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:22.762757 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:22.762757 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:22.762757 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:22.762757 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:22.762757 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:22.762757 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:22.762757 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:22.762757 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:22.762757 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:15:22.762757 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:15:22.762757 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:15:22.762757 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:15:22.763741 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:15:22.763741 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:15:22.763741 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:15:22.763741 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:15:22.763741 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:22.763741 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:22.763741 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:22.763741 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:22.763741 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:22.763741 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:22.763741 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:22.763741 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:22.763741 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:22.763741 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:22.763741 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:22.763741 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:22.764589 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:22.764589 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:22.764589 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:22.764589 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:22.764589 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:15:22.764589 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:15:22.764589 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:15:22.764589 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:15:22.764589 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:15:22.764589 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:15:22.764589 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:15:22.764589 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:22.768847 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 310/624] 10.222.0.33 () {56 vars in 1102 bytes} [Fri Jul 15 19:15:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:22.769679 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2f45fd-b47f-4aa2-9b4c-b531aeee34c7 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:22.805958 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af7c7a9-d6d0-4c52-8fb4-7045729738f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:22.807940 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af7c7a9-d6d0-4c52-8fb4-7045729738f8 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:22.808813 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af7c7a9-d6d0-4c52-8fb4-7045729738f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:22.809245 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af7c7a9-d6d0-4c52-8fb4-7045729738f8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:22.824253 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af7c7a9-d6d0-4c52-8fb4-7045729738f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:15:22.834383 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-9af7c7a9-d6d0-4c52-8fb4-7045729738f8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:15:22.834383 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:22.834383 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:22.834383 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:22.834383 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:22.834383 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:22.834383 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:22.834383 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:22.834383 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:22.834383 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:22.834383 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:22.834383 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:22.834383 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:15:22.834383 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:15:22.834383 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:15:22.834383 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:15:22.835567 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:15:22.835567 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:15:22.835567 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:15:22.835567 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:15:22.835567 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:22.835567 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:22.835567 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:22.835567 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:22.835567 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:22.835567 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:22.835567 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:22.835567 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:22.835567 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:22.835567 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:22.835567 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:22.835567 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:22.836627 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:22.836627 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:22.836627 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:22.836627 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:22.836627 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:15:22.836627 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:15:22.836627 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:15:22.836627 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:15:22.836627 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:15:22.836627 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:15:22.836627 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:15:22.836627 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:22.839471 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 315/625] 10.222.0.33 () {56 vars in 1102 bytes} [Fri Jul 15 19:15:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:22.840467 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af7c7a9-d6d0-4c52-8fb4-7045729738f8 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:22.875919 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:22.895725 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2f45fd-b47f-4aa2-9b4c-b531aeee34c7 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:22.932281 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690d3916-8d52-44e1-ac3d-e8d49851824d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:22.933313 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d3916-8d52-44e1-ac3d-e8d49851824d None admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:22.933660 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d3916-8d52-44e1-ac3d-e8d49851824d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:22.934047 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d3916-8d52-44e1-ac3d-e8d49851824d None admin] PATH_INFO: `/v3/projects/55a756ccaef14cb49b26b0d68dd0b466/users/ef76f7fe39a3499aaeebf85c176804f3/roles/70de044d6962417593b6d03571e7502e` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:22.937165 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:22.952106 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690d3916-8d52-44e1-ac3d-e8d49851824d None admin] RBAC: Authorizing `identity:create_grant(project_id=55a756ccaef14cb49b26b0d68dd0b466, user_id=ef76f7fe39a3499aaeebf85c176804f3, role_id=70de044d6962417593b6d03571e7502e)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:22.953727 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690d3916-8d52-44e1-ac3d-e8d49851824d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:22.966431 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af7c7a9-d6d0-4c52-8fb4-7045729738f8 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:23.002267 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c19f75f-7007-46d2-bd93-a09807d9a713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:23.002855 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c19f75f-7007-46d2-bd93-a09807d9a713 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:23.003135 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c19f75f-7007-46d2-bd93-a09807d9a713 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:23.003408 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c19f75f-7007-46d2-bd93-a09807d9a713 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:23.004148 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c19f75f-7007-46d2-bd93-a09807d9a713 None admin] RBAC: Adding query filter params (name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:23.004456 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c19f75f-7007-46d2-bd93-a09807d9a713 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:23.006410 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c19f75f-7007-46d2-bd93-a09807d9a713 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:23.019414 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 316/626] 10.222.0.33 () {56 vars in 1103 bytes} [Fri Jul 15 19:15:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:23.019414 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c19f75f-7007-46d2-bd93-a09807d9a713 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:23.081457 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 311/627] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jul 15 19:15:22 2022] PUT /identity/v3/projects/55a756ccaef14cb49b26b0d68dd0b466/users/ef76f7fe39a3499aaeebf85c176804f3/roles/70de044d6962417593b6d03571e7502e => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:15:23.090103 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690d3916-8d52-44e1-ac3d-e8d49851824d None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:23.181374 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c19f75f-7007-46d2-bd93-a09807d9a713 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:23.241960 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710db4a3-de17-4900-a85d-eb868bf97732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:23.243220 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710db4a3-de17-4900-a85d-eb868bf97732 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:23.243771 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710db4a3-de17-4900-a85d-eb868bf97732 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:23.244324 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710db4a3-de17-4900-a85d-eb868bf97732 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:23.246113 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710db4a3-de17-4900-a85d-eb868bf97732 None admin] RBAC: Adding query filter params (name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:23.246432 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710db4a3-de17-4900-a85d-eb868bf97732 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:23.248406 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710db4a3-de17-4900-a85d-eb868bf97732 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:23.251929 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:23.261996 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 317/628] 10.222.0.33 () {56 vars in 1103 bytes} [Fri Jul 15 19:15:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:23.263388 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdb4100-de3c-412e-b752-65e7a1ebb80a None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:23.263823 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdb4100-de3c-412e-b752-65e7a1ebb80a None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:23.265000 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdb4100-de3c-412e-b752-65e7a1ebb80a None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:23.271575 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 318/629] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:15:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:15:23.272751 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdb4100-de3c-412e-b752-65e7a1ebb80a None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:23.286337 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690d3916-8d52-44e1-ac3d-e8d49851824d None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:23.324649 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:23.332032 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e88e534-1757-49a8-a438-5a36b4f63707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:23.332032 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e88e534-1757-49a8-a438-5a36b4f63707 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:23.332032 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e88e534-1757-49a8-a438-5a36b4f63707 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:23.332032 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e88e534-1757-49a8-a438-5a36b4f63707 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:23.342389 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e88e534-1757-49a8-a438-5a36b4f63707 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:23.343905 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e88e534-1757-49a8-a438-5a36b4f63707 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:23.362545 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-5e88e534-1757-49a8-a438-5a36b4f63707 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:23.362545 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:23.362545 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:23.362545 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:23.362545 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:23.362545 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:23.362545 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:23.362545 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:23.362545 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:23.362545 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:23.362545 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:23.362545 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:23.362545 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:15:23.362545 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:15:23.362545 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:15:23.368512 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:15:23.368512 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:23.368512 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:23.368512 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:23.368512 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:23.368512 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:23.368512 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:23.368512 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:23.368512 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:23.368512 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:23.368512 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:23.368512 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:23.368512 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:23.368512 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:23.368512 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:23.368512 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:23.368512 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:23.369798 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:15:23.369798 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:15:23.369798 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:23.369798 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:23.369798 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 312/630] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:15:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:23.369798 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542da972-5b9c-4ce9-8b09-a4add8067829 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:23.369798 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542da972-5b9c-4ce9-8b09-a4add8067829 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:23.369798 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542da972-5b9c-4ce9-8b09-a4add8067829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:23.388174 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-542da972-5b9c-4ce9-8b09-a4add8067829 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:15:23.389405 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-542da972-5b9c-4ce9-8b09-a4add8067829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:15:23.421938 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdb4100-de3c-412e-b752-65e7a1ebb80a None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:23.467486 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bb57de-3822-4b30-b71f-1d9f75361263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:23.468301 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb57de-3822-4b30-b71f-1d9f75361263 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:23.468716 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb57de-3822-4b30-b71f-1d9f75361263 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:23.468824 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb57de-3822-4b30-b71f-1d9f75361263 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:23.469069 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 313/631] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:15:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:15:23.470254 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542da972-5b9c-4ce9-8b09-a4add8067829 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:23.496017 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bb57de-3822-4b30-b71f-1d9f75361263 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:23.497246 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bb57de-3822-4b30-b71f-1d9f75361263 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:23.522905 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-c7bb57de-3822-4b30-b71f-1d9f75361263 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:23.522905 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:23.522905 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:23.522905 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:23.522905 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:23.522905 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:23.522905 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:23.522905 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:23.522905 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:23.522905 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:23.522905 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:23.522905 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:23.522905 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:15:23.522905 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:15:23.522905 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:15:23.523835 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:15:23.523835 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:23.523835 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:23.523835 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:23.523835 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:23.523835 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:23.523835 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:23.523835 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:23.523835 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:23.523835 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:23.523835 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:23.523835 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:23.523835 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:23.523835 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:23.523835 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:23.523835 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:23.523835 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:23.525463 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:15:23.525463 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:15:23.525463 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:23.525463 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:23.525463 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 319/632] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:15:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:23.525463 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bb57de-3822-4b30-b71f-1d9f75361263 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:23.593473 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:23.595341 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542da972-5b9c-4ce9-8b09-a4add8067829 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:23.610995 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:23.633477 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd16e6c-eecc-4eb7-844d-76a022b825a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:23.634184 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd16e6c-eecc-4eb7-844d-76a022b825a6 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:23.634691 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd16e6c-eecc-4eb7-844d-76a022b825a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:23.635026 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd16e6c-eecc-4eb7-844d-76a022b825a6 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:23.635766 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd16e6c-eecc-4eb7-844d-76a022b825a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:23.636074 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd16e6c-eecc-4eb7-844d-76a022b825a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:23.637456 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd16e6c-eecc-4eb7-844d-76a022b825a6 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:23.650481 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 314/633] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:15:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:23.651405 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd16e6c-eecc-4eb7-844d-76a022b825a6 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:23.655412 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bb57de-3822-4b30-b71f-1d9f75361263 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:23.698769 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46596d9-b083-4a87-a2de-dc7cec68bd8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:23.700193 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46596d9-b083-4a87-a2de-dc7cec68bd8d None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:23.701138 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46596d9-b083-4a87-a2de-dc7cec68bd8d None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:23.701757 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46596d9-b083-4a87-a2de-dc7cec68bd8d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:23.716927 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46596d9-b083-4a87-a2de-dc7cec68bd8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:15:23.728656 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-c46596d9-b083-4a87-a2de-dc7cec68bd8d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:15:23.728656 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:23.728656 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:23.728656 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:23.728656 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:23.728656 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:23.728656 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:23.728656 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:23.728656 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:23.728656 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:23.728656 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:23.728656 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:23.728656 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:15:23.728656 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:15:23.728656 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:15:23.728656 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:15:23.729613 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:15:23.729613 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:15:23.729613 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:15:23.729613 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:15:23.729613 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:23.729613 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:23.729613 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:23.729613 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:23.729613 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:23.729613 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:23.729613 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:23.729613 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:23.729613 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:23.729613 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:23.729613 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:23.729613 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:23.730506 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:23.730506 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:23.730506 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:23.730506 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:23.730506 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:15:23.730506 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:15:23.730506 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:15:23.730506 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:15:23.730506 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:15:23.730506 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:15:23.730506 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:15:23.730506 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:23.734236 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 320/634] 10.222.0.33 () {56 vars in 1102 bytes} [Fri Jul 15 19:15:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:23.735398 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46596d9-b083-4a87-a2de-dc7cec68bd8d None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:23.765807 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd16e6c-eecc-4eb7-844d-76a022b825a6 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:23.820286 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d7112c-e95f-4f0f-8add-66de4674ac0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:23.821864 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d7112c-e95f-4f0f-8add-66de4674ac0c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:23.822643 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d7112c-e95f-4f0f-8add-66de4674ac0c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:23.823128 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d7112c-e95f-4f0f-8add-66de4674ac0c None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:23.824863 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d7112c-e95f-4f0f-8add-66de4674ac0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:23.825726 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d7112c-e95f-4f0f-8add-66de4674ac0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:23.828047 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d7112c-e95f-4f0f-8add-66de4674ac0c None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:23.846027 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 315/635] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:15:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:23.847639 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d7112c-e95f-4f0f-8add-66de4674ac0c None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:23.872918 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46596d9-b083-4a87-a2de-dc7cec68bd8d None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:23.884687 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:23.895786 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:23.910702 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4319c703-d862-4afd-a5e7-c5dc41593f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:23.911634 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4319c703-d862-4afd-a5e7-c5dc41593f45 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:23.912004 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4319c703-d862-4afd-a5e7-c5dc41593f45 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:23.912349 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4319c703-d862-4afd-a5e7-c5dc41593f45 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:23.920872 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4319c703-d862-4afd-a5e7-c5dc41593f45 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:23.922477 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4319c703-d862-4afd-a5e7-c5dc41593f45 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:23.937705 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-4319c703-d862-4afd-a5e7-c5dc41593f45 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 15 19:15:23.937705 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:23.937705 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:23.937705 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:23.937705 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:23.937705 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:23.937705 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:23.937705 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:23.937705 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:23.937705 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:23.937705 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:23.937705 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:23.937705 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 19:15:23.937705 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 19:15:23.937705 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 19:15:23.937705 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 19:15:23.940960 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:23.940960 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:23.940960 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:15:23.940960 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:15:23.940960 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:15:23.940960 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:15:23.940960 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:23.940960 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:23.940960 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:23.940960 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:23.940960 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:23.940960 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:23.940960 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:23.940960 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:23.940960 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:23.940960 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:23.940960 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:23.940960 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:23.941979 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:23.941979 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:23.941979 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 19:15:23.941979 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 19:15:23.941979 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 19:15:23.941979 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 19:15:23.941979 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 19:15:23.941979 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 19:15:23.941979 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 15 19:15:23.941979 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:23.941979 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 321/636] 10.222.0.33 () {56 vars in 1093 bytes} [Fri Jul 15 19:15:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:23.941979 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4319c703-d862-4afd-a5e7-c5dc41593f45 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:23.991485 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d7112c-e95f-4f0f-8add-66de4674ac0c None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:24.030375 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c784230f-5262-4fb1-ac54-af5d82172f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:24.032264 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c784230f-5262-4fb1-ac54-af5d82172f9e None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:24.032779 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c784230f-5262-4fb1-ac54-af5d82172f9e None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:24.033263 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c784230f-5262-4fb1-ac54-af5d82172f9e None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:24.034256 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c784230f-5262-4fb1-ac54-af5d82172f9e None admin] RBAC: Adding query filter params (name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:24.034772 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c784230f-5262-4fb1-ac54-af5d82172f9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:24.037505 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c784230f-5262-4fb1-ac54-af5d82172f9e None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:24.054283 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 316/637] 10.222.0.33 () {56 vars in 1103 bytes} [Fri Jul 15 19:15:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:24.054857 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c784230f-5262-4fb1-ac54-af5d82172f9e None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:24.068419 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4319c703-d862-4afd-a5e7-c5dc41593f45 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:24.104527 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c6baac-44d1-4d88-8ee6-9262fbedadb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:24.105324 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c6baac-44d1-4d88-8ee6-9262fbedadb3 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:24.105609 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c6baac-44d1-4d88-8ee6-9262fbedadb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:24.105917 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c6baac-44d1-4d88-8ee6-9262fbedadb3 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:24.113552 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c6baac-44d1-4d88-8ee6-9262fbedadb3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:24.115546 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c6baac-44d1-4d88-8ee6-9262fbedadb3 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:24.131271 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-98c6baac-44d1-4d88-8ee6-9262fbedadb3 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 15 19:15:24.131271 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:24.131271 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:24.131271 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:24.131271 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:24.131271 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:24.131271 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:24.131271 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:24.131271 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:24.131271 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:24.131271 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:24.131271 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:24.131271 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 19:15:24.131271 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 19:15:24.131271 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 19:15:24.131271 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 19:15:24.132277 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:24.132277 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:24.132277 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:15:24.132277 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:15:24.132277 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:15:24.132277 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:15:24.132277 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:24.132277 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:24.132277 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:24.132277 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:24.132277 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:24.132277 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:24.132277 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:24.132277 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:24.132277 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:24.132277 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:24.132277 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:24.132277 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:24.133251 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:24.133251 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:24.133251 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 19:15:24.133251 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 19:15:24.133251 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 19:15:24.133251 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 19:15:24.133251 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 19:15:24.133251 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 19:15:24.133251 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 15 19:15:24.133251 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:24.133251 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 322/638] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 15 19:15:23 2022] GET /identity/v3/users/glance => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:24.133251 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c6baac-44d1-4d88-8ee6-9262fbedadb3 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:24.156184 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:24.162416 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c784230f-5262-4fb1-ac54-af5d82172f9e None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:24.164683 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:24.199008 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d54c75e-d618-4455-88f4-c07bec314270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:24.199676 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d54c75e-d618-4455-88f4-c07bec314270 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:24.199956 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d54c75e-d618-4455-88f4-c07bec314270 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:24.200227 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d54c75e-d618-4455-88f4-c07bec314270 None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:24.201067 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d54c75e-d618-4455-88f4-c07bec314270 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:24.201373 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d54c75e-d618-4455-88f4-c07bec314270 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:24.202969 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d54c75e-d618-4455-88f4-c07bec314270 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:24.221325 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 317/639] 10.222.0.33 () {56 vars in 1133 bytes} [Fri Jul 15 19:15:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:24.222754 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9007dd-e60e-4c81-9a06-2d8f90112b06 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:24.223046 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9007dd-e60e-4c81-9a06-2d8f90112b06 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:24.223324 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9007dd-e60e-4c81-9a06-2d8f90112b06 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:24.225026 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 318/640] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:15:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:15:24.225875 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9007dd-e60e-4c81-9a06-2d8f90112b06 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:24.239305 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c6baac-44d1-4d88-8ee6-9262fbedadb3 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:24.276253 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d391dd-c11e-4dcb-8cbb-84ec35641db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:24.277048 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d391dd-c11e-4dcb-8cbb-84ec35641db4 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:24.277334 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d391dd-c11e-4dcb-8cbb-84ec35641db4 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:24.277874 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d391dd-c11e-4dcb-8cbb-84ec35641db4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:24.288195 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d391dd-c11e-4dcb-8cbb-84ec35641db4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:24.289637 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d391dd-c11e-4dcb-8cbb-84ec35641db4 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:24.308180 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-23d391dd-c11e-4dcb-8cbb-84ec35641db4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:24.308180 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:24.308180 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:24.308180 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:24.308180 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:24.308180 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:24.308180 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:24.308180 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:24.308180 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:24.308180 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:24.308180 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:24.308180 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:24.308180 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:15:24.308180 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:15:24.308180 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:15:24.309204 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:15:24.309204 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:24.309204 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:24.309204 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:24.309204 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:24.309204 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:24.309204 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:24.309204 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:24.309204 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:24.309204 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:24.309204 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:24.309204 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:24.309204 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:24.309204 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:24.309204 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:24.309204 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:24.309204 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:24.310529 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:15:24.310529 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:15:24.310529 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:24.310529 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:24.310529 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 323/641] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:15:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:24.310529 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d391dd-c11e-4dcb-8cbb-84ec35641db4 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:24.349572 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9007dd-e60e-4c81-9a06-2d8f90112b06 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:24.386769 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31287104-cca4-45bb-9dca-9f4087a9519c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:24.387441 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31287104-cca4-45bb-9dca-9f4087a9519c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:24.387725 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31287104-cca4-45bb-9dca-9f4087a9519c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:24.388065 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31287104-cca4-45bb-9dca-9f4087a9519c None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:24.388919 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31287104-cca4-45bb-9dca-9f4087a9519c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:24.389227 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31287104-cca4-45bb-9dca-9f4087a9519c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:24.390755 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31287104-cca4-45bb-9dca-9f4087a9519c None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:24.403805 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:24.410137 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 319/642] 10.222.0.33 () {56 vars in 1137 bytes} [Fri Jul 15 19:15:24 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:24.415191 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5536481-d6b6-4ab8-95ad-7302c67cd3c4 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:24.415191 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5536481-d6b6-4ab8-95ad-7302c67cd3c4 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:24.415191 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5536481-d6b6-4ab8-95ad-7302c67cd3c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:24.430708 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-d5536481-d6b6-4ab8-95ad-7302c67cd3c4 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:15:24.431243 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d391dd-c11e-4dcb-8cbb-84ec35641db4 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:24.431905 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d5536481-d6b6-4ab8-95ad-7302c67cd3c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:15:24.440007 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:24.467689 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454c7a5a-c987-4339-b2b6-2f4bc2862531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:24.468348 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454c7a5a-c987-4339-b2b6-2f4bc2862531 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:24.468627 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454c7a5a-c987-4339-b2b6-2f4bc2862531 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:24.468910 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454c7a5a-c987-4339-b2b6-2f4bc2862531 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:24.508776 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 320/643] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:15:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:15:24.509662 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5536481-d6b6-4ab8-95ad-7302c67cd3c4 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:24.550782 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454c7a5a-c987-4339-b2b6-2f4bc2862531 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:24.552819 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454c7a5a-c987-4339-b2b6-2f4bc2862531 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:24.617320 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5536481-d6b6-4ab8-95ad-7302c67cd3c4 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:24.642732 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:24.652895 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8469d5f8-330f-4d3f-b43c-61c2f4942e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:24.653566 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8469d5f8-330f-4d3f-b43c-61c2f4942e66 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:24.653878 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8469d5f8-330f-4d3f-b43c-61c2f4942e66 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:24.655377 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8469d5f8-330f-4d3f-b43c-61c2f4942e66 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:24.655377 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8469d5f8-330f-4d3f-b43c-61c2f4942e66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:24.655377 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8469d5f8-330f-4d3f-b43c-61c2f4942e66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:24.656529 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8469d5f8-330f-4d3f-b43c-61c2f4942e66 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:24.666123 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 324/644] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 15 19:15:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 19:15:24.667400 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454c7a5a-c987-4339-b2b6-2f4bc2862531 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:24.671752 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 321/645] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:15:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:24.672741 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8469d5f8-330f-4d3f-b43c-61c2f4942e66 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:24.679016 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:24.770944 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454c7a5a-c987-4339-b2b6-2f4bc2862531 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:24.785677 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8469d5f8-330f-4d3f-b43c-61c2f4942e66 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:24.818046 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd36dce-cbf8-4e4d-a67c-66f13c8ee0a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:24.818698 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd36dce-cbf8-4e4d-a67c-66f13c8ee0a9 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:24.818976 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd36dce-cbf8-4e4d-a67c-66f13c8ee0a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:24.819245 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd36dce-cbf8-4e4d-a67c-66f13c8ee0a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:24.828164 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65c0054-95e8-43ff-810f-488b7bb81b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:24.828966 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65c0054-95e8-43ff-810f-488b7bb81b03 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:24.829248 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65c0054-95e8-43ff-810f-488b7bb81b03 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:24.829523 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65c0054-95e8-43ff-810f-488b7bb81b03 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:24.840811 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65c0054-95e8-43ff-810f-488b7bb81b03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:15:24.848399 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-e65c0054-95e8-43ff-810f-488b7bb81b03 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:15:24.848399 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:24.848399 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:24.848399 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:24.848399 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:24.848399 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:24.848399 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:24.848399 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:24.848399 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:24.848399 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:24.848399 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:24.848399 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:24.848399 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:15:24.848399 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:15:24.848399 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:15:24.848399 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:15:24.849929 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:15:24.849929 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:15:24.849929 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:15:24.849929 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:15:24.849929 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:24.849929 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:24.849929 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:24.849929 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:24.849929 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:24.849929 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:24.849929 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:24.849929 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:24.849929 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:24.849929 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:24.849929 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:24.849929 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:24.850949 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:24.850949 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:24.850949 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:24.850949 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:24.850949 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:15:24.850949 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:15:24.850949 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:15:24.850949 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:15:24.850949 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:15:24.850949 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:15:24.850949 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:15:24.850949 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:24.850949 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 322/646] 10.222.0.33 () {56 vars in 1102 bytes} [Fri Jul 15 19:15:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:24.851695 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65c0054-95e8-43ff-810f-488b7bb81b03 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:24.877536 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:24.899559 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd36dce-cbf8-4e4d-a67c-66f13c8ee0a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:24.902052 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd36dce-cbf8-4e4d-a67c-66f13c8ee0a9 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:24.923291 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:24.966620 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65c0054-95e8-43ff-810f-488b7bb81b03 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:25.002172 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d32288f-c808-4483-a56a-40441b543b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:25.003069 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d32288f-c808-4483-a56a-40441b543b71 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:25.003346 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d32288f-c808-4483-a56a-40441b543b71 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:25.003730 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d32288f-c808-4483-a56a-40441b543b71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:25.013496 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 325/647] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 15 19:15:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 19:15:25.014588 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd36dce-cbf8-4e4d-a67c-66f13c8ee0a9 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:25.019400 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d32288f-c808-4483-a56a-40441b543b71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:25.021384 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d32288f-c808-4483-a56a-40441b543b71 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:25.039845 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-5d32288f-c808-4483-a56a-40441b543b71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:25.039845 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:25.039845 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:25.039845 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:25.039845 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:25.039845 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:25.039845 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:25.039845 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:25.039845 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:25.039845 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:25.039845 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:25.039845 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:25.039845 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:15:25.039845 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:15:25.039845 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:15:25.040787 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:15:25.040787 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:25.040787 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:25.040787 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:25.040787 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:25.040787 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:25.040787 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:25.040787 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:25.040787 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:25.040787 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:25.040787 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:25.040787 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:25.040787 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:25.040787 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:25.040787 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:25.040787 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:25.040787 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:25.041690 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:15:25.041690 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:15:25.041690 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:25.041690 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:25.041690 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 323/648] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:15:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:25.042432 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d32288f-c808-4483-a56a-40441b543b71 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:25.110233 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:25.130451 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd36dce-cbf8-4e4d-a67c-66f13c8ee0a9 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:25.147628 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d32288f-c808-4483-a56a-40441b543b71 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:25.162515 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:25.168555 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749560e7-b8d8-4de3-ab32-eea79735572a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:25.169350 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749560e7-b8d8-4de3-ab32-eea79735572a None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:25.169630 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749560e7-b8d8-4de3-ab32-eea79735572a None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:25.169931 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749560e7-b8d8-4de3-ab32-eea79735572a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:25.177446 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749560e7-b8d8-4de3-ab32-eea79735572a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:25.178915 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749560e7-b8d8-4de3-ab32-eea79735572a None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:25.182482 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc627f6-bb1f-4c0b-9eac-0b4631aaf4e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:25.183152 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc627f6-bb1f-4c0b-9eac-0b4631aaf4e7 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:25.183436 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc627f6-bb1f-4c0b-9eac-0b4631aaf4e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:25.183710 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc627f6-bb1f-4c0b-9eac-0b4631aaf4e7 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:25.184465 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc627f6-bb1f-4c0b-9eac-0b4631aaf4e7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:25.184770 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc627f6-bb1f-4c0b-9eac-0b4631aaf4e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:25.186161 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc627f6-bb1f-4c0b-9eac-0b4631aaf4e7 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:25.194095 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-749560e7-b8d8-4de3-ab32-eea79735572a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 15 19:15:25.194095 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:25.194095 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:25.194095 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:25.194095 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:25.194095 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:25.194095 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:25.194095 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:25.194095 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:25.194095 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:25.194095 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:25.194095 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:25.194095 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 19:15:25.194095 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 19:15:25.194095 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 19:15:25.194095 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 19:15:25.200004 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:25.200004 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:25.200004 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:15:25.200004 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:15:25.200004 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:15:25.200004 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:15:25.200004 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:25.200004 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:25.200004 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:25.200004 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:25.200004 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:25.200004 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:25.200004 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:25.200004 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:25.200004 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:25.200004 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:25.200004 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:25.200004 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:25.200972 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:25.200972 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:25.200972 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 19:15:25.200972 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 19:15:25.200972 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 19:15:25.200972 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 19:15:25.200972 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 19:15:25.200972 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 19:15:25.200972 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 15 19:15:25.200972 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:25.202080 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 324/649] 10.222.0.33 () {56 vars in 1103 bytes} [Fri Jul 15 19:15:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:25.202080 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc627f6-bb1f-4c0b-9eac-0b4631aaf4e7 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:25.202235 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 326/650] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 15 19:15:25 2022] GET /identity/v3/users/cinder => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:25.202235 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749560e7-b8d8-4de3-ab32-eea79735572a None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:25.300427 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc627f6-bb1f-4c0b-9eac-0b4631aaf4e7 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:25.301954 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749560e7-b8d8-4de3-ab32-eea79735572a None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:25.334065 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddc1ea3-b3a4-4cc3-919c-e49cfca90c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:25.334862 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc1ea3-b3a4-4cc3-919c-e49cfca90c4a None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:25.335142 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc1ea3-b3a4-4cc3-919c-e49cfca90c4a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:25.335416 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc1ea3-b3a4-4cc3-919c-e49cfca90c4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:25.336153 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cfca62-0cf6-40e9-bc91-c50bdff78f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:25.336807 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cfca62-0cf6-40e9-bc91-c50bdff78f3b None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:25.337101 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cfca62-0cf6-40e9-bc91-c50bdff78f3b None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:25.337377 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cfca62-0cf6-40e9-bc91-c50bdff78f3b None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:25.338137 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cfca62-0cf6-40e9-bc91-c50bdff78f3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:25.338448 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cfca62-0cf6-40e9-bc91-c50bdff78f3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:25.339944 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cfca62-0cf6-40e9-bc91-c50bdff78f3b None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:25.345439 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddc1ea3-b3a4-4cc3-919c-e49cfca90c4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:25.346902 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddc1ea3-b3a4-4cc3-919c-e49cfca90c4a None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:25.351473 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 327/651] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:15:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:25.352277 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cfca62-0cf6-40e9-bc91-c50bdff78f3b None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:25.354826 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:25.364691 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-fddc1ea3-b3a4-4cc3-919c-e49cfca90c4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:25.364691 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:25.364691 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:25.364691 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:25.364691 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:25.364691 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:25.364691 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:25.364691 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:25.364691 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:25.364691 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:25.364691 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:25.364691 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:25.364691 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:15:25.364691 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:15:25.364691 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:15:25.366157 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:15:25.366157 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:25.366157 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:25.366157 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:25.366157 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:25.366157 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:25.366157 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:25.366157 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:25.366157 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:25.366157 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:25.366157 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:25.366157 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:25.366157 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:25.366157 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:25.366157 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:25.366157 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:25.366157 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:25.367248 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:15:25.367248 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:15:25.367248 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:25.367248 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:25.367248 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 325/652] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:15:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:25.367248 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddc1ea3-b3a4-4cc3-919c-e49cfca90c4a None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:25.394375 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:25.453791 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cfca62-0cf6-40e9-bc91-c50bdff78f3b None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:25.465083 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddc1ea3-b3a4-4cc3-919c-e49cfca90c4a None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:25.489279 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492c8b83-92b8-4c4b-beb2-260d9dea9c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:25.490200 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492c8b83-92b8-4c4b-beb2-260d9dea9c18 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:25.490484 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492c8b83-92b8-4c4b-beb2-260d9dea9c18 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:25.490762 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492c8b83-92b8-4c4b-beb2-260d9dea9c18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:25.500702 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f6183e-83b9-4c81-8d89-ac903f7f02a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:25.501358 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492c8b83-92b8-4c4b-beb2-260d9dea9c18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:25.501474 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f6183e-83b9-4c81-8d89-ac903f7f02a8 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:25.501636 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f6183e-83b9-4c81-8d89-ac903f7f02a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:25.501937 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f6183e-83b9-4c81-8d89-ac903f7f02a8 None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:25.502771 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f6183e-83b9-4c81-8d89-ac903f7f02a8 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:25.502864 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492c8b83-92b8-4c4b-beb2-260d9dea9c18 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:25.503079 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f6183e-83b9-4c81-8d89-ac903f7f02a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:25.504497 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f6183e-83b9-4c81-8d89-ac903f7f02a8 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:25.520738 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-492c8b83-92b8-4c4b-beb2-260d9dea9c18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:25.520738 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:25.520738 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:25.520738 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:25.520738 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:25.520738 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:25.520738 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:25.520738 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:25.520738 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:25.520738 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:25.520738 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:25.520738 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:25.520738 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:15:25.520738 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:15:25.520738 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:15:25.522637 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:15:25.522637 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:25.522637 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:25.522637 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:25.522637 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:25.522637 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:25.522637 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:25.522637 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:25.522637 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:25.522637 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:25.522637 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:25.522637 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:25.522637 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:25.522637 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:25.522637 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:25.522637 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:25.522637 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:25.523894 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:15:25.523894 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:15:25.523894 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:25.523894 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:25.524118 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 326/653] 10.222.0.33 () {56 vars in 1137 bytes} [Fri Jul 15 19:15:25 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:25.524357 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 328/654] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:15:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:25.524357 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492c8b83-92b8-4c4b-beb2-260d9dea9c18 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:25.525373 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90c90d6-f71b-431c-8d89-cb2de6fc371d None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:25.525663 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90c90d6-f71b-431c-8d89-cb2de6fc371d None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:25.525975 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90c90d6-f71b-431c-8d89-cb2de6fc371d None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:25.527393 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 327/655] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:15:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:15:25.528090 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90c90d6-f71b-431c-8d89-cb2de6fc371d None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:25.583116 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:25.613362 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:25.628186 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492c8b83-92b8-4c4b-beb2-260d9dea9c18 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:25.630623 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90c90d6-f71b-431c-8d89-cb2de6fc371d None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:25.661402 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd9579f-c535-4ec7-bfa7-39fe15c7ae32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:25.662257 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd9579f-c535-4ec7-bfa7-39fe15c7ae32 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:25.662699 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd9579f-c535-4ec7-bfa7-39fe15c7ae32 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:25.662938 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd9579f-c535-4ec7-bfa7-39fe15c7ae32 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:25.664506 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7096dc-c81f-48c2-8704-0ab69d886b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:25.665157 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7096dc-c81f-48c2-8704-0ab69d886b8e None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:25.665432 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7096dc-c81f-48c2-8704-0ab69d886b8e None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:25.665703 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7096dc-c81f-48c2-8704-0ab69d886b8e None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:25.666447 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7096dc-c81f-48c2-8704-0ab69d886b8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:25.666744 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7096dc-c81f-48c2-8704-0ab69d886b8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:25.668097 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7096dc-c81f-48c2-8704-0ab69d886b8e None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:25.670384 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd9579f-c535-4ec7-bfa7-39fe15c7ae32 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:25.671873 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd9579f-c535-4ec7-bfa7-39fe15c7ae32 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:25.679785 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 328/656] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:15:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:25.680569 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7096dc-c81f-48c2-8704-0ab69d886b8e None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:25.686974 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-9cd9579f-c535-4ec7-bfa7-39fe15c7ae32 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:15:25.686974 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:25.686974 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:25.686974 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:25.686974 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:25.686974 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:25.686974 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:25.686974 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:25.686974 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:25.686974 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:25.686974 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:25.686974 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:25.686974 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 19:15:25.686974 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 19:15:25.686974 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 19:15:25.688029 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 19:15:25.688029 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:25.688029 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:25.688029 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:25.688029 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:25.688029 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:25.688029 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:25.688029 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:25.688029 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:25.688029 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:25.688029 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:25.688029 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:25.688029 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:25.688029 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:25.688029 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:25.688029 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:25.688029 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:25.688948 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 19:15:25.688948 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 19:15:25.688948 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:15:25.688948 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:15:25.688948 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:15:25.688948 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:15:25.688948 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:15:25.688948 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:25.688948 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 329/657] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 15 19:15:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:25.689496 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd9579f-c535-4ec7-bfa7-39fe15c7ae32 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:25.789146 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd9579f-c535-4ec7-bfa7-39fe15c7ae32 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:25.793318 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7096dc-c81f-48c2-8704-0ab69d886b8e None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:25.804373 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:25.825060 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89a3bdb-a926-4f93-9b3d-4ff4979b140f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:25.825710 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89a3bdb-a926-4f93-9b3d-4ff4979b140f None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:25.826010 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89a3bdb-a926-4f93-9b3d-4ff4979b140f None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:25.826347 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89a3bdb-a926-4f93-9b3d-4ff4979b140f None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:25.826996 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89a3bdb-a926-4f93-9b3d-4ff4979b140f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:25.827294 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89a3bdb-a926-4f93-9b3d-4ff4979b140f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:25.827778 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a86e7b6-4a79-4d46-a2e2-e1936e480159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:25.828431 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a86e7b6-4a79-4d46-a2e2-e1936e480159 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:25.828646 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89a3bdb-a926-4f93-9b3d-4ff4979b140f None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:25.828736 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a86e7b6-4a79-4d46-a2e2-e1936e480159 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:25.828983 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a86e7b6-4a79-4d46-a2e2-e1936e480159 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:25.842150 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 330/658] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:15:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:25.843630 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a8217-9ab3-4f04-bc32-cf3cfa6e84b0 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:25.844059 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a8217-9ab3-4f04-bc32-cf3cfa6e84b0 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:25.844486 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a8217-9ab3-4f04-bc32-cf3cfa6e84b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:25.844629 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:25.861113 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-ba4a8217-9ab3-4f04-bc32-cf3cfa6e84b0 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:15:25.862326 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-ba4a8217-9ab3-4f04-bc32-cf3cfa6e84b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:15:25.897736 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a86e7b6-4a79-4d46-a2e2-e1936e480159 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:25.899215 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a86e7b6-4a79-4d46-a2e2-e1936e480159 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:25.929389 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 331/659] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:15:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:15:25.930188 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4a8217-9ab3-4f04-bc32-cf3cfa6e84b0 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:26.029000 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 329/660] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 15 19:15:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 19:15:26.029892 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a86e7b6-4a79-4d46-a2e2-e1936e480159 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:26.032189 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4a8217-9ab3-4f04-bc32-cf3cfa6e84b0 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:26.035250 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:26.066743 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6151c251-b9c8-4653-851a-68c86169eeda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:26.067528 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6151c251-b9c8-4653-851a-68c86169eeda None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:26.067810 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6151c251-b9c8-4653-851a-68c86169eeda None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:26.068085 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6151c251-b9c8-4653-851a-68c86169eeda None admin] PATH_INFO: `/v3/projects/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:26.074654 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6151c251-b9c8-4653-851a-68c86169eeda None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:26.076099 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6151c251-b9c8-4653-851a-68c86169eeda None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:26.076812 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:26.090841 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-6151c251-b9c8-4653-851a-68c86169eeda None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:15:26.090841 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:26.090841 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:26.090841 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:26.090841 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:26.090841 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:26.090841 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:26.090841 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:26.090841 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:26.090841 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:26.090841 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:26.090841 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:26.090841 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 19:15:26.090841 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 19:15:26.090841 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 19:15:26.091903 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 19:15:26.091903 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:26.091903 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:26.091903 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:26.091903 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:26.091903 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:26.091903 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:26.091903 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:26.091903 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:26.091903 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:26.091903 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:26.091903 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:26.091903 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:26.091903 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:26.091903 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:26.091903 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:26.091903 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:26.092807 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 19:15:26.092807 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 19:15:26.092807 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:15:26.092807 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:15:26.092807 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:15:26.092807 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:15:26.092807 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:15:26.092807 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:26.092807 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 332/661] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 15 19:15:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:26.092807 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6151c251-b9c8-4653-851a-68c86169eeda None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:26.131049 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a86e7b6-4a79-4d46-a2e2-e1936e480159 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:26.165828 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c4c8c7-e6a1-4aeb-baf3-eb8b6c93332f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:26.166487 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c4c8c7-e6a1-4aeb-baf3-eb8b6c93332f None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:26.166771 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c4c8c7-e6a1-4aeb-baf3-eb8b6c93332f None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:26.167044 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c4c8c7-e6a1-4aeb-baf3-eb8b6c93332f None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:26.167838 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c4c8c7-e6a1-4aeb-baf3-eb8b6c93332f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:26.168144 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c4c8c7-e6a1-4aeb-baf3-eb8b6c93332f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:26.169589 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c4c8c7-e6a1-4aeb-baf3-eb8b6c93332f None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:26.181109 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 330/662] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jul 15 19:15:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:26.181864 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c4c8c7-e6a1-4aeb-baf3-eb8b6c93332f None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:26.198358 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6151c251-b9c8-4653-851a-68c86169eeda None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:26.232255 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b459108-aaea-440a-aa53-6093b058f706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:26.233046 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b459108-aaea-440a-aa53-6093b058f706 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:26.233337 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b459108-aaea-440a-aa53-6093b058f706 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:26.233610 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b459108-aaea-440a-aa53-6093b058f706 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:26.241172 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b459108-aaea-440a-aa53-6093b058f706 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:26.242628 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b459108-aaea-440a-aa53-6093b058f706 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:26.258214 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-0b459108-aaea-440a-aa53-6093b058f706 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 15 19:15:26.258214 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:26.258214 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:26.258214 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:26.258214 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:26.258214 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:26.258214 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:26.258214 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:26.258214 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:26.258214 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:26.258214 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:26.258214 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:26.258214 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 19:15:26.258214 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 19:15:26.258214 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 19:15:26.258214 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 19:15:26.259407 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:26.259407 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:26.259407 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:15:26.259407 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:15:26.259407 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:15:26.259407 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:15:26.259407 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:26.259407 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:26.259407 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:26.259407 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:26.259407 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:26.259407 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:26.259407 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:26.259407 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:26.259407 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:26.259407 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:26.259407 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:26.259407 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:26.260722 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:26.260722 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:26.260722 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 19:15:26.260722 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 19:15:26.260722 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 19:15:26.260722 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 19:15:26.260722 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 19:15:26.260722 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 19:15:26.260722 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 15 19:15:26.260722 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:26.260722 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 333/663] 10.222.0.33 () {56 vars in 1102 bytes} [Fri Jul 15 19:15:26 2022] GET /identity/v3/users/neutron => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:26.260722 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b459108-aaea-440a-aa53-6093b058f706 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:26.285317 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:26.288467 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c4c8c7-e6a1-4aeb-baf3-eb8b6c93332f None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:26.295847 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:26.323585 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2d5137-a24e-4ec7-bb5f-be4b1e15a175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:26.324401 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2d5137-a24e-4ec7-bb5f-be4b1e15a175 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:26.324667 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2d5137-a24e-4ec7-bb5f-be4b1e15a175 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:26.324944 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2d5137-a24e-4ec7-bb5f-be4b1e15a175 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:26.335872 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2d5137-a24e-4ec7-bb5f-be4b1e15a175 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:15:26.343590 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-ba2d5137-a24e-4ec7-bb5f-be4b1e15a175 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:15:26.343590 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:26.343590 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:26.343590 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:26.343590 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:26.343590 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:26.343590 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:26.343590 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:26.343590 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:26.343590 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:26.343590 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:26.343590 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:26.343590 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:15:26.343590 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:15:26.343590 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:15:26.343590 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:15:26.344549 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:15:26.344549 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:15:26.344549 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:15:26.344549 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:15:26.344549 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:26.344549 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:26.344549 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:26.344549 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:26.344549 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:26.344549 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:26.344549 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:26.344549 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:26.344549 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:26.344549 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:26.344549 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:26.344549 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:26.345426 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:26.345426 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:26.345426 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:26.345426 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:26.345426 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:15:26.345426 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:15:26.345426 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:15:26.345426 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:15:26.345426 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:15:26.345426 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:15:26.345426 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:15:26.345426 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:26.346429 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 331/664] 10.222.0.33 () {56 vars in 1102 bytes} [Fri Jul 15 19:15:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:26.346713 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2d5137-a24e-4ec7-bb5f-be4b1e15a175 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:26.366382 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b459108-aaea-440a-aa53-6093b058f706 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:26.400689 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e2e2f4-6070-4583-b463-ff4cb8366c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:26.401475 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2e2f4-6070-4583-b463-ff4cb8366c77 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:26.401786 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2e2f4-6070-4583-b463-ff4cb8366c77 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:26.402063 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2e2f4-6070-4583-b463-ff4cb8366c77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:26.412039 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e2e2f4-6070-4583-b463-ff4cb8366c77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:26.413510 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e2e2f4-6070-4583-b463-ff4cb8366c77 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:26.431988 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-30e2e2f4-6070-4583-b463-ff4cb8366c77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:26.431988 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:26.431988 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:26.431988 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:26.431988 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:26.431988 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:26.431988 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:26.431988 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:26.431988 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:26.431988 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:26.431988 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:26.431988 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:26.431988 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:15:26.431988 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:15:26.431988 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:15:26.433063 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:15:26.433063 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:26.433063 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:26.433063 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:26.433063 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:26.433063 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:26.433063 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:26.433063 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:26.433063 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:26.433063 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:26.433063 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:26.433063 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:26.433063 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:26.433063 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:26.433063 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:26.433063 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:26.433063 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:26.434036 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:15:26.434036 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:15:26.434036 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:26.434036 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:26.434036 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 334/665] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:15:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:26.434313 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e2e2f4-6070-4583-b463-ff4cb8366c77 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:26.448278 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2d5137-a24e-4ec7-bb5f-be4b1e15a175 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:26.481538 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6d07f0-4c94-4c80-a5eb-79e04ee137fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:26.482219 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6d07f0-4c94-4c80-a5eb-79e04ee137fa None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:26.482497 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6d07f0-4c94-4c80-a5eb-79e04ee137fa None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:26.482765 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6d07f0-4c94-4c80-a5eb-79e04ee137fa None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:26.483545 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6d07f0-4c94-4c80-a5eb-79e04ee137fa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:26.483849 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6d07f0-4c94-4c80-a5eb-79e04ee137fa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:26.485286 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6d07f0-4c94-4c80-a5eb-79e04ee137fa None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:26.496847 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 332/666] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jul 15 19:15:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:26.497610 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6d07f0-4c94-4c80-a5eb-79e04ee137fa None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:26.510261 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:26.529521 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:26.537391 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e2e2f4-6070-4583-b463-ff4cb8366c77 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:26.573002 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f191d43-1b26-4768-b517-f48a78386cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:26.573651 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f191d43-1b26-4768-b517-f48a78386cf9 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:26.573962 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f191d43-1b26-4768-b517-f48a78386cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:26.574237 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f191d43-1b26-4768-b517-f48a78386cf9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:26.575072 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f191d43-1b26-4768-b517-f48a78386cf9 None admin] RBAC: Adding query filter params (role.id=70de044d6962417593b6d03571e7502e, scope.project.id=55a756ccaef14cb49b26b0d68dd0b466, user.id=7673a3b5c0324e07a9535ac70ed7a491) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:26.575390 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f191d43-1b26-4768-b517-f48a78386cf9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:26.576812 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f191d43-1b26-4768-b517-f48a78386cf9 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:26.582654 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 335/667] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 15 19:15:26 2022] GET /identity/v3/role_assignments?user.id=7673a3b5c0324e07a9535ac70ed7a491&scope.project.id=55a756ccaef14cb49b26b0d68dd0b466&role.id=70de044d6962417593b6d03571e7502e => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:26.583954 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f191d43-1b26-4768-b517-f48a78386cf9 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:26.602178 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6d07f0-4c94-4c80-a5eb-79e04ee137fa None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:26.642293 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f5d4b2-662d-4ce5-a4b7-dc280f3c70f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:26.642989 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f5d4b2-662d-4ce5-a4b7-dc280f3c70f4 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:26.643275 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f5d4b2-662d-4ce5-a4b7-dc280f3c70f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:26.643553 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f5d4b2-662d-4ce5-a4b7-dc280f3c70f4 None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:26.644424 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f5d4b2-662d-4ce5-a4b7-dc280f3c70f4 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:26.644739 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f5d4b2-662d-4ce5-a4b7-dc280f3c70f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:26.646289 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f5d4b2-662d-4ce5-a4b7-dc280f3c70f4 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:26.669930 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 333/668] 10.222.0.33 () {56 vars in 1139 bytes} [Fri Jul 15 19:15:26 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:26.669930 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f5d4b2-662d-4ce5-a4b7-dc280f3c70f4 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:26.720395 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f191d43-1b26-4768-b517-f48a78386cf9 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:26.764698 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7157e426-cd5e-4379-9526-da59c814821e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:26.764913 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:26.765873 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7157e426-cd5e-4379-9526-da59c814821e None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:26.766140 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7157e426-cd5e-4379-9526-da59c814821e None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:26.766417 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7157e426-cd5e-4379-9526-da59c814821e None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:26.767198 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7157e426-cd5e-4379-9526-da59c814821e None admin] RBAC: Adding query filter params (name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:26.767506 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7157e426-cd5e-4379-9526-da59c814821e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:26.768869 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7157e426-cd5e-4379-9526-da59c814821e None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:26.783469 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 336/669] 10.222.0.33 () {56 vars in 1103 bytes} [Fri Jul 15 19:15:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:26.783469 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7157e426-cd5e-4379-9526-da59c814821e None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:26.783469 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:26.789198 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f5d4b2-662d-4ce5-a4b7-dc280f3c70f4 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:26.835469 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2aa9b83-88d8-428e-8b4c-521add6e08f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:26.836468 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aa9b83-88d8-428e-8b4c-521add6e08f1 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:26.836823 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aa9b83-88d8-428e-8b4c-521add6e08f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:26.837096 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aa9b83-88d8-428e-8b4c-521add6e08f1 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:26.837779 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2aa9b83-88d8-428e-8b4c-521add6e08f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:26.838112 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2aa9b83-88d8-428e-8b4c-521add6e08f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:26.839654 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2aa9b83-88d8-428e-8b4c-521add6e08f1 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:26.853933 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 334/670] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:15:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:26.853933 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2aa9b83-88d8-428e-8b4c-521add6e08f1 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:26.901932 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7157e426-cd5e-4379-9526-da59c814821e None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:26.936984 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e68071c-cbe4-4eef-ae55-cfbace12942c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:26.937638 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e68071c-cbe4-4eef-ae55-cfbace12942c None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:26.937929 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e68071c-cbe4-4eef-ae55-cfbace12942c None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:26.938203 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e68071c-cbe4-4eef-ae55-cfbace12942c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:26.939024 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e68071c-cbe4-4eef-ae55-cfbace12942c None admin] RBAC: Adding query filter params (role.id=70de044d6962417593b6d03571e7502e, scope.project.id=55a756ccaef14cb49b26b0d68dd0b466, user.id=04b5b06b0015450a95a28bbcc757cc8a) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:26.939330 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e68071c-cbe4-4eef-ae55-cfbace12942c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:26.940733 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e68071c-cbe4-4eef-ae55-cfbace12942c None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:26.957817 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 337/671] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 15 19:15:26 2022] GET /identity/v3/role_assignments?user.id=04b5b06b0015450a95a28bbcc757cc8a&scope.project.id=55a756ccaef14cb49b26b0d68dd0b466&role.id=70de044d6962417593b6d03571e7502e => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:26.957817 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e68071c-cbe4-4eef-ae55-cfbace12942c None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:26.978616 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2aa9b83-88d8-428e-8b4c-521add6e08f1 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:27.016972 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa91ba15-ac52-4aae-9a3b-c9beeeae8f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:27.017748 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa91ba15-ac52-4aae-9a3b-c9beeeae8f17 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:27.018136 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa91ba15-ac52-4aae-9a3b-c9beeeae8f17 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:27.018485 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa91ba15-ac52-4aae-9a3b-c9beeeae8f17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:27.030466 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:27.037544 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:27.081877 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e68071c-cbe4-4eef-ae55-cfbace12942c None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:27.106259 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa91ba15-ac52-4aae-9a3b-c9beeeae8f17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:27.107889 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa91ba15-ac52-4aae-9a3b-c9beeeae8f17 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:27.135350 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b83d302-e26b-426d-9352-d1cb3853d841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:27.136162 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b83d302-e26b-426d-9352-d1cb3853d841 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:27.136449 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b83d302-e26b-426d-9352-d1cb3853d841 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:27.136728 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b83d302-e26b-426d-9352-d1cb3853d841 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:27.151653 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b83d302-e26b-426d-9352-d1cb3853d841 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:27.153117 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b83d302-e26b-426d-9352-d1cb3853d841 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:27.176758 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-7b83d302-e26b-426d-9352-d1cb3853d841 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:27.176758 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:27.176758 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:27.176758 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:27.176758 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:27.176758 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:27.176758 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:27.176758 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:27.176758 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:27.176758 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:27.176758 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:27.176758 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:27.176758 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:15:27.176758 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:15:27.176758 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:15:27.178412 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:15:27.178412 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:27.178412 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:27.178412 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:27.178412 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:27.178412 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:27.178412 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:27.178412 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:27.178412 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:27.178412 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:27.178412 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:27.178412 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:27.178412 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:27.178412 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:27.178412 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:27.178412 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:27.178412 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:27.179322 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:15:27.179322 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:15:27.179322 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:27.179322 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:27.179322 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 338/672] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:15:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:27.180174 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b83d302-e26b-426d-9352-d1cb3853d841 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:27.246858 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 335/673] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 15 19:15:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 19:15:27.248038 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa91ba15-ac52-4aae-9a3b-c9beeeae8f17 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:27.307141 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b83d302-e26b-426d-9352-d1cb3853d841 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:27.308441 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:27.331318 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:27.355182 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f307644-39be-48be-8b66-c524b92653ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:27.356008 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f307644-39be-48be-8b66-c524b92653ee None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:27.356315 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f307644-39be-48be-8b66-c524b92653ee None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:27.356591 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f307644-39be-48be-8b66-c524b92653ee None admin] PATH_INFO: `/v3/projects/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:27.366264 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f307644-39be-48be-8b66-c524b92653ee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:27.368334 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f307644-39be-48be-8b66-c524b92653ee None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:27.370519 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa91ba15-ac52-4aae-9a3b-c9beeeae8f17 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:27.392287 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-0f307644-39be-48be-8b66-c524b92653ee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:15:27.392287 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:27.392287 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:27.392287 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:27.392287 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:27.392287 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:27.392287 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:27.392287 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:27.392287 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:27.392287 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:27.392287 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:27.392287 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:27.392287 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 19:15:27.392287 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 19:15:27.392287 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 19:15:27.393251 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 19:15:27.393251 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:27.393251 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:27.393251 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:27.393251 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:27.393251 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:27.393251 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:27.393251 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:27.393251 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:27.393251 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:27.393251 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:27.393251 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:27.393251 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:27.393251 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:27.393251 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:27.393251 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:27.393251 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:27.394211 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 19:15:27.394211 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 19:15:27.394211 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:15:27.394211 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:15:27.394211 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:15:27.394211 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:15:27.394211 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:15:27.394211 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:27.394211 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 339/674] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 15 19:15:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:27.395212 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f307644-39be-48be-8b66-c524b92653ee None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:27.410768 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126b8086-4b55-4595-8a37-d80283fd075f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:27.412395 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b8086-4b55-4595-8a37-d80283fd075f None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:27.412885 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b8086-4b55-4595-8a37-d80283fd075f None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:27.413414 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b8086-4b55-4595-8a37-d80283fd075f None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:27.414494 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126b8086-4b55-4595-8a37-d80283fd075f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:27.415098 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126b8086-4b55-4595-8a37-d80283fd075f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:27.416695 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126b8086-4b55-4595-8a37-d80283fd075f None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:27.429804 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 336/675] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:15:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:27.430998 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126b8086-4b55-4595-8a37-d80283fd075f None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:27.532016 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f307644-39be-48be-8b66-c524b92653ee None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:27.539747 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126b8086-4b55-4595-8a37-d80283fd075f None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:27.543106 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:27.573819 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4ce4ab-955c-4feb-bcc1-277929a51455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:27.575151 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4ce4ab-955c-4feb-bcc1-277929a51455 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:27.575669 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4ce4ab-955c-4feb-bcc1-277929a51455 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:27.576225 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4ce4ab-955c-4feb-bcc1-277929a51455 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:27.576526 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c2254b-17b3-4e53-8290-d0f1a682b2ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:27.577192 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c2254b-17b3-4e53-8290-d0f1a682b2ba None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:27.577686 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c2254b-17b3-4e53-8290-d0f1a682b2ba None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:27.578011 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c2254b-17b3-4e53-8290-d0f1a682b2ba None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:27.578859 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c2254b-17b3-4e53-8290-d0f1a682b2ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:27.579535 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c2254b-17b3-4e53-8290-d0f1a682b2ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:27.581650 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c2254b-17b3-4e53-8290-d0f1a682b2ba None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:27.591826 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4ce4ab-955c-4feb-bcc1-277929a51455 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:27.593707 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4ce4ab-955c-4feb-bcc1-277929a51455 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:27.599040 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 337/676] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jul 15 19:15:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:27.600122 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c2254b-17b3-4e53-8290-d0f1a682b2ba None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:27.613951 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-1a4ce4ab-955c-4feb-bcc1-277929a51455 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:27.613951 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:27.613951 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:27.613951 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:27.613951 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:27.613951 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:27.613951 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:27.613951 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:27.613951 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:27.613951 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:27.613951 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:27.613951 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:27.613951 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:15:27.613951 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:15:27.613951 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:15:27.614966 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:15:27.614966 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:27.614966 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:27.614966 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:27.614966 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:27.614966 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:27.614966 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:27.614966 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:27.614966 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:27.614966 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:27.614966 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:27.614966 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:27.614966 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:27.614966 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:27.614966 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:27.614966 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:27.614966 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:27.615883 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:15:27.615883 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:15:27.615883 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:27.615883 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:27.621881 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 340/677] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:15:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:27.623048 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4ce4ab-955c-4feb-bcc1-277929a51455 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:27.657380 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:27.723323 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c2254b-17b3-4e53-8290-d0f1a682b2ba None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:27.736136 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4ce4ab-955c-4feb-bcc1-277929a51455 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:27.765642 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa56d01-ea33-4e47-97b0-6baa0b914673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:27.766998 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa56d01-ea33-4e47-97b0-6baa0b914673 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:27.767884 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa56d01-ea33-4e47-97b0-6baa0b914673 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:27.768252 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa56d01-ea33-4e47-97b0-6baa0b914673 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:27.780998 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930bd763-9402-4bde-9cb2-9c381c0bd8fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:27.781950 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930bd763-9402-4bde-9cb2-9c381c0bd8fe None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:27.782451 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa56d01-ea33-4e47-97b0-6baa0b914673 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:27.782572 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930bd763-9402-4bde-9cb2-9c381c0bd8fe None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:27.782987 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930bd763-9402-4bde-9cb2-9c381c0bd8fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:27.783994 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930bd763-9402-4bde-9cb2-9c381c0bd8fe None admin] RBAC: Adding query filter params (role.id=70de044d6962417593b6d03571e7502e, scope.project.id=55a756ccaef14cb49b26b0d68dd0b466, user.id=c86c2df3d378490ca7ea497a73c19d40) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:27.784121 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa56d01-ea33-4e47-97b0-6baa0b914673 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:27.784565 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930bd763-9402-4bde-9cb2-9c381c0bd8fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:27.787020 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930bd763-9402-4bde-9cb2-9c381c0bd8fe None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:27.794343 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 341/678] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 15 19:15:27 2022] GET /identity/v3/role_assignments?user.id=c86c2df3d378490ca7ea497a73c19d40&scope.project.id=55a756ccaef14cb49b26b0d68dd0b466&role.id=70de044d6962417593b6d03571e7502e => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:27.795512 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930bd763-9402-4bde-9cb2-9c381c0bd8fe None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:27.804920 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-9aa56d01-ea33-4e47-97b0-6baa0b914673 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 15 19:15:27.804920 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:27.804920 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:27.804920 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:27.804920 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:27.804920 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:27.804920 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:27.804920 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:27.804920 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:27.804920 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:27.804920 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:27.804920 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:27.804920 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 19:15:27.804920 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 19:15:27.804920 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 19:15:27.804920 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 19:15:27.806643 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:27.806643 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:27.806643 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:15:27.806643 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:15:27.806643 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:15:27.806643 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:15:27.806643 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:27.806643 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:27.806643 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:27.806643 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:27.806643 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:27.806643 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:27.806643 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:27.806643 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:27.806643 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:27.806643 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:27.806643 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:27.806643 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:27.809506 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:27.809506 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:27.809506 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 19:15:27.809506 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 19:15:27.809506 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 19:15:27.809506 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 19:15:27.809506 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 19:15:27.809506 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 19:15:27.809506 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 15 19:15:27.809506 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:27.818066 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 338/679] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jul 15 19:15:27 2022] GET /identity/v3/users/swift => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:27.818985 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa56d01-ea33-4e47-97b0-6baa0b914673 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:27.852764 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:27.980672 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:27.988918 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930bd763-9402-4bde-9cb2-9c381c0bd8fe None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:28.009360 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa56d01-ea33-4e47-97b0-6baa0b914673 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:28.032613 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd22098-56e4-4b42-85f3-b4d71a642360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:28.033560 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd22098-56e4-4b42-85f3-b4d71a642360 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:28.034103 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd22098-56e4-4b42-85f3-b4d71a642360 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:28.034710 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd22098-56e4-4b42-85f3-b4d71a642360 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:28.035615 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd22098-56e4-4b42-85f3-b4d71a642360 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:28.036104 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd22098-56e4-4b42-85f3-b4d71a642360 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:28.038280 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd22098-56e4-4b42-85f3-b4d71a642360 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:28.050648 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580b2add-fc58-4204-bd1b-900a7ffffef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:28.050980 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 342/680] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:15:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:28.055673 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b2add-fc58-4204-bd1b-900a7ffffef8 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:28.055968 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b2add-fc58-4204-bd1b-900a7ffffef8 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:28.056250 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b2add-fc58-4204-bd1b-900a7ffffef8 None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:28.057109 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580b2add-fc58-4204-bd1b-900a7ffffef8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:28.057424 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580b2add-fc58-4204-bd1b-900a7ffffef8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:28.058917 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580b2add-fc58-4204-bd1b-900a7ffffef8 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:28.061908 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd22098-56e4-4b42-85f3-b4d71a642360 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:28.080199 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 339/681] 10.222.0.33 () {56 vars in 1135 bytes} [Fri Jul 15 19:15:27 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:28.087969 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580b2add-fc58-4204-bd1b-900a7ffffef8 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:28.190764 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd22098-56e4-4b42-85f3-b4d71a642360 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:28.201751 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:28.230416 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580b2add-fc58-4204-bd1b-900a7ffffef8 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:28.233578 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23cea6f-7f21-4c85-bd17-e89e82a0c75f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:28.234500 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23cea6f-7f21-4c85-bd17-e89e82a0c75f None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:28.234783 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23cea6f-7f21-4c85-bd17-e89e82a0c75f None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:28.235056 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23cea6f-7f21-4c85-bd17-e89e82a0c75f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:28.243022 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23cea6f-7f21-4c85-bd17-e89e82a0c75f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:28.244592 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23cea6f-7f21-4c85-bd17-e89e82a0c75f None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:28.262499 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-b23cea6f-7f21-4c85-bd17-e89e82a0c75f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:15:28.262499 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:28.262499 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:28.262499 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:28.262499 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:28.262499 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:28.262499 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:28.262499 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:28.262499 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:28.262499 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:28.262499 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:28.262499 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:28.262499 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 19:15:28.262499 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 19:15:28.262499 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 19:15:28.263442 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 19:15:28.263442 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:28.263442 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:28.263442 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:28.263442 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:28.263442 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:28.263442 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:28.263442 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:28.263442 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:28.263442 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:28.263442 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:28.263442 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:28.263442 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:28.263442 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:28.263442 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:28.263442 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:28.263442 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:28.264355 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 19:15:28.264355 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 19:15:28.264355 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:15:28.264355 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:15:28.264355 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:15:28.264355 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:15:28.264355 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:15:28.264355 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:28.264355 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 343/682] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 15 19:15:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:15:28.270197 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23cea6f-7f21-4c85-bd17-e89e82a0c75f None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:28.275506 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6edf353-fd85-4f7f-97ed-57ac2e8e2039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:28.276172 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6edf353-fd85-4f7f-97ed-57ac2e8e2039 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:28.276457 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6edf353-fd85-4f7f-97ed-57ac2e8e2039 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:28.276736 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6edf353-fd85-4f7f-97ed-57ac2e8e2039 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:28.290822 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:28.377303 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6edf353-fd85-4f7f-97ed-57ac2e8e2039 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:28.379155 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6edf353-fd85-4f7f-97ed-57ac2e8e2039 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:28.387449 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23cea6f-7f21-4c85-bd17-e89e82a0c75f None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:28.424821 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc8d061-0dd0-4887-8247-dd5a132d2405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:28.425508 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc8d061-0dd0-4887-8247-dd5a132d2405 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:28.425806 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc8d061-0dd0-4887-8247-dd5a132d2405 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:28.426091 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc8d061-0dd0-4887-8247-dd5a132d2405 None admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:28.426980 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc8d061-0dd0-4887-8247-dd5a132d2405 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:28.427302 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc8d061-0dd0-4887-8247-dd5a132d2405 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:28.428808 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc8d061-0dd0-4887-8247-dd5a132d2405 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:28.445944 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 344/683] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jul 15 19:15:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:28.448570 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc8d061-0dd0-4887-8247-dd5a132d2405 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:28.468474 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:28.506712 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 340/684] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 15 19:15:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 19:15:28.512290 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6edf353-fd85-4f7f-97ed-57ac2e8e2039 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:28.532823 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:28.559773 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc8d061-0dd0-4887-8247-dd5a132d2405 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:28.596677 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe687a1-1627-49ee-b197-f7d8c1dd52ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:28.597339 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe687a1-1627-49ee-b197-f7d8c1dd52ab None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:28.597624 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe687a1-1627-49ee-b197-f7d8c1dd52ab None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:28.597948 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe687a1-1627-49ee-b197-f7d8c1dd52ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:28.599369 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe687a1-1627-49ee-b197-f7d8c1dd52ab None admin] RBAC: Adding query filter params (role.id=70de044d6962417593b6d03571e7502e, scope.project.id=55a756ccaef14cb49b26b0d68dd0b466, user.id=b2a283d7d3594e37943bda4529d0a775) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:28.599704 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe687a1-1627-49ee-b197-f7d8c1dd52ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:28.601124 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe687a1-1627-49ee-b197-f7d8c1dd52ab None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:28.613872 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 345/685] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 15 19:15:28 2022] GET /identity/v3/role_assignments?user.id=b2a283d7d3594e37943bda4529d0a775&scope.project.id=55a756ccaef14cb49b26b0d68dd0b466&role.id=70de044d6962417593b6d03571e7502e => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:28.697604 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6edf353-fd85-4f7f-97ed-57ac2e8e2039 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:28.750987 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e46b5c3-e0b7-4a41-a9c1-f68c51e253d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:28.752411 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e46b5c3-e0b7-4a41-a9c1-f68c51e253d4 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:28.752902 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e46b5c3-e0b7-4a41-a9c1-f68c51e253d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:28.753398 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e46b5c3-e0b7-4a41-a9c1-f68c51e253d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:28.771442 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e46b5c3-e0b7-4a41-a9c1-f68c51e253d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:28.773956 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e46b5c3-e0b7-4a41-a9c1-f68c51e253d4 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:28.793889 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-5e46b5c3-e0b7-4a41-a9c1-f68c51e253d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:28.793889 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:28.793889 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:28.793889 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:28.793889 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:28.793889 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:28.793889 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:28.793889 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:28.793889 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:28.793889 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:28.793889 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:28.793889 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:28.793889 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:15:28.793889 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:15:28.793889 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:15:28.794871 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:15:28.794871 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:28.794871 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:28.794871 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:28.794871 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:28.794871 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:28.794871 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:28.794871 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:28.794871 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:28.794871 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:28.794871 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:28.794871 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:28.794871 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:28.794871 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:28.794871 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:28.794871 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:28.794871 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:28.795776 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:15:28.795776 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:15:28.795776 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:28.795776 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:28.795776 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 341/686] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:15:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:15:28.802614 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe687a1-1627-49ee-b197-f7d8c1dd52ab None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:28.889164 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aaed53-ec86-4f86-bf58-dc47f86e518a None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:28.889482 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aaed53-ec86-4f86-bf58-dc47f86e518a None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:28.890030 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aaed53-ec86-4f86-bf58-dc47f86e518a None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:28.892884 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 342/687] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:15:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:15:28.898927 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d7607e-0362-49b5-af2b-5cbb2f7000f9 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:28.899206 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d7607e-0362-49b5-af2b-5cbb2f7000f9 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:28.899485 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d7607e-0362-49b5-af2b-5cbb2f7000f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:28.923696 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-b0d7607e-0362-49b5-af2b-5cbb2f7000f9 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:15:28.925959 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b0d7607e-0362-49b5-af2b-5cbb2f7000f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:15:28.933038 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe687a1-1627-49ee-b197-f7d8c1dd52ab None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:28.967298 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:28.991230 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a4175c-5922-4768-bd8e-852ef0c8bb39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:28.991913 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a4175c-5922-4768-bd8e-852ef0c8bb39 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:28.992199 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a4175c-5922-4768-bd8e-852ef0c8bb39 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:28.992473 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a4175c-5922-4768-bd8e-852ef0c8bb39 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:28.993207 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a4175c-5922-4768-bd8e-852ef0c8bb39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:28.993511 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a4175c-5922-4768-bd8e-852ef0c8bb39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:28.994917 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a4175c-5922-4768-bd8e-852ef0c8bb39 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:29.011078 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:29.024263 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 346/688] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:15:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:29.029587 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a4175c-5922-4768-bd8e-852ef0c8bb39 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:29.031016 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 343/689] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:15:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:15:29.054233 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d7607e-0362-49b5-af2b-5cbb2f7000f9 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:29.174810 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a4175c-5922-4768-bd8e-852ef0c8bb39 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:29.213632 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d7607e-0362-49b5-af2b-5cbb2f7000f9 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:29.245621 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a56212-91fc-4bde-8129-f894d47ab592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:29.246447 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a56212-91fc-4bde-8129-f894d47ab592 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:29.246728 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a56212-91fc-4bde-8129-f894d47ab592 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:29.247001 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a56212-91fc-4bde-8129-f894d47ab592 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:29.261308 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a56212-91fc-4bde-8129-f894d47ab592 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:29.262782 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a56212-91fc-4bde-8129-f894d47ab592 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:29.279203 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-02a56212-91fc-4bde-8129-f894d47ab592 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:15:29.279203 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:29.279203 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:29.279203 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:29.279203 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:29.279203 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:29.279203 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:29.279203 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:29.279203 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:29.279203 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:29.279203 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:29.279203 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:29.279203 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 19:15:29.279203 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 19:15:29.279203 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 19:15:29.280147 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 19:15:29.280147 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:29.280147 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:29.280147 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:29.280147 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:29.280147 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:29.280147 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:29.280147 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:29.280147 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:29.280147 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:29.280147 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:29.280147 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:29.280147 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:29.280147 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:29.280147 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:29.280147 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:29.280147 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:29.281019 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 19:15:29.281019 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 19:15:29.281019 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:15:29.281019 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:15:29.281019 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:15:29.281019 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:15:29.281019 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:15:29.281019 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:29.281019 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 347/690] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 15 19:15:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:29.286987 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a56212-91fc-4bde-8129-f894d47ab592 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:29.314511 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd2833c-2360-48ca-a5d4-ba3044df3a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:29.315330 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd2833c-2360-48ca-a5d4-ba3044df3a66 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:29.315614 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd2833c-2360-48ca-a5d4-ba3044df3a66 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:29.315889 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd2833c-2360-48ca-a5d4-ba3044df3a66 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:29.329178 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd2833c-2360-48ca-a5d4-ba3044df3a66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:15:29.345145 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-6fd2833c-2360-48ca-a5d4-ba3044df3a66 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:15:29.345145 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:29.345145 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:29.345145 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:29.345145 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:29.345145 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:29.345145 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:29.345145 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:29.345145 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:29.345145 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:29.345145 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:29.345145 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:29.345145 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:15:29.345145 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:15:29.345145 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:15:29.345145 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:15:29.346176 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:15:29.346176 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:15:29.346176 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:15:29.346176 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:15:29.346176 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:29.346176 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:29.346176 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:29.346176 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:29.346176 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:29.346176 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:29.346176 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:29.346176 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:29.346176 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:29.346176 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:29.346176 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:29.346176 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:29.347119 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:29.347119 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:29.347119 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:29.347119 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:29.347119 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:15:29.347119 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:15:29.347119 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:15:29.347119 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:15:29.347119 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:15:29.347119 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:15:29.347119 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:15:29.347119 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:29.347119 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 344/691] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jul 15 19:15:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:29.353266 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd2833c-2360-48ca-a5d4-ba3044df3a66 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:29.362121 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:29.402560 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:29.439938 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a56212-91fc-4bde-8129-f894d47ab592 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:29.491835 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd2833c-2360-48ca-a5d4-ba3044df3a66 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:29.529602 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8cc775-1a7d-462e-b82f-db2fd6d3a39b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:29.530353 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8cc775-1a7d-462e-b82f-db2fd6d3a39b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:29.530605 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8cc775-1a7d-462e-b82f-db2fd6d3a39b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:29.530879 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8cc775-1a7d-462e-b82f-db2fd6d3a39b None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:29.530981 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1dff82-fc15-4fc1-81e3-45e3edd5015f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:29.531630 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8cc775-1a7d-462e-b82f-db2fd6d3a39b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:29.531731 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1dff82-fc15-4fc1-81e3-45e3edd5015f None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:29.531906 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1dff82-fc15-4fc1-81e3-45e3edd5015f None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:29.532012 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8cc775-1a7d-462e-b82f-db2fd6d3a39b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:29.532169 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1dff82-fc15-4fc1-81e3-45e3edd5015f None admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:29.532967 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1dff82-fc15-4fc1-81e3-45e3edd5015f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:29.533273 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1dff82-fc15-4fc1-81e3-45e3edd5015f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:29.533368 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8cc775-1a7d-462e-b82f-db2fd6d3a39b None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:29.536315 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1dff82-fc15-4fc1-81e3-45e3edd5015f None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:29.543551 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 345/692] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 15 19:15:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:29.545666 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bdd929-89c1-43a3-b070-a7619fb6c14c None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:29.545985 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bdd929-89c1-43a3-b070-a7619fb6c14c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:29.546266 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bdd929-89c1-43a3-b070-a7619fb6c14c None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:29.547753 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 346/694] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:15:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:15:29.547934 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 348/694] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jul 15 19:15:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:29.555358 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bdd929-89c1-43a3-b070-a7619fb6c14c None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:29.555504 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c6c1b3-7d8d-4ce3-bf2f-29782dd0fb8e None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:29.555504 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c6c1b3-7d8d-4ce3-bf2f-29782dd0fb8e None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:29.555790 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c6c1b3-7d8d-4ce3-bf2f-29782dd0fb8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:29.587070 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-59c6c1b3-7d8d-4ce3-bf2f-29782dd0fb8e None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:15:29.588472 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-59c6c1b3-7d8d-4ce3-bf2f-29782dd0fb8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:15:29.666504 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:29.673920 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 349/695] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:15:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:15:29.675154 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c6c1b3-7d8d-4ce3-bf2f-29782dd0fb8e None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:29.689720 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bdd929-89c1-43a3-b070-a7619fb6c14c None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:29.732166 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f1c76d-a3db-4fd9-a352-bef9ce622f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:29.733854 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f1c76d-a3db-4fd9-a352-bef9ce622f7b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:29.734350 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f1c76d-a3db-4fd9-a352-bef9ce622f7b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:29.734866 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f1c76d-a3db-4fd9-a352-bef9ce622f7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:29.746953 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f1c76d-a3db-4fd9-a352-bef9ce622f7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:29.748768 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f1c76d-a3db-4fd9-a352-bef9ce622f7b None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:29.762188 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:29.767801 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-00f1c76d-a3db-4fd9-a352-bef9ce622f7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:29.767801 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:29.767801 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:29.767801 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:29.767801 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:29.767801 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:29.767801 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:29.767801 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:29.767801 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:29.767801 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:29.767801 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:29.767801 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:29.767801 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:15:29.767801 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:15:29.767801 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:15:29.768688 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:15:29.768688 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:29.768688 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:29.768688 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:29.768688 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:29.768688 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:29.768688 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:29.768688 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:29.768688 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:29.768688 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:29.768688 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:29.768688 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:29.768688 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:29.768688 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:29.768688 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:29.768688 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:29.768688 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:29.769608 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:15:29.769608 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:15:29.769608 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:29.769608 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:29.776235 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 347/696] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:15:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:29.776235 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f1c76d-a3db-4fd9-a352-bef9ce622f7b None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:29.816386 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c6c1b3-7d8d-4ce3-bf2f-29782dd0fb8e None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:29.855926 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9fbae9-6862-4f82-8fd1-810006ccd31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:29.857039 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9fbae9-6862-4f82-8fd1-810006ccd31f None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:29.857530 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9fbae9-6862-4f82-8fd1-810006ccd31f None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:29.858061 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9fbae9-6862-4f82-8fd1-810006ccd31f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:29.859275 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9fbae9-6862-4f82-8fd1-810006ccd31f None admin] RBAC: Adding query filter params (role.id=70de044d6962417593b6d03571e7502e, scope.project.id=55a756ccaef14cb49b26b0d68dd0b466, user.id=ef76f7fe39a3499aaeebf85c176804f3) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:29.859796 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9fbae9-6862-4f82-8fd1-810006ccd31f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:29.861558 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9fbae9-6862-4f82-8fd1-810006ccd31f None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:29.869009 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 350/697] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 15 19:15:29 2022] GET /identity/v3/role_assignments?user.id=ef76f7fe39a3499aaeebf85c176804f3&scope.project.id=55a756ccaef14cb49b26b0d68dd0b466&role.id=70de044d6962417593b6d03571e7502e => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:29.870275 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9fbae9-6862-4f82-8fd1-810006ccd31f None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:29.893188 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f1c76d-a3db-4fd9-a352-bef9ce622f7b None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:29.932449 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1547824d-4bd8-4e3e-a6c6-bf9f545b0513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:29.933269 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1547824d-4bd8-4e3e-a6c6-bf9f545b0513 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:29.933555 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1547824d-4bd8-4e3e-a6c6-bf9f545b0513 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:29.933876 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1547824d-4bd8-4e3e-a6c6-bf9f545b0513 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:29.942225 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:29.948348 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1547824d-4bd8-4e3e-a6c6-bf9f545b0513 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:29.950179 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1547824d-4bd8-4e3e-a6c6-bf9f545b0513 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:29.970645 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-1547824d-4bd8-4e3e-a6c6-bf9f545b0513 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:29.970645 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:29.970645 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:29.970645 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:29.970645 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:29.970645 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:29.970645 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:29.970645 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:29.970645 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:29.970645 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:29.970645 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:29.970645 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:29.970645 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:15:29.970645 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:15:29.970645 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:15:29.971618 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:15:29.971618 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:29.971618 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:29.971618 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:29.971618 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:29.971618 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:29.971618 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:29.971618 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:29.971618 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:29.971618 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:29.971618 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:29.971618 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:29.971618 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:29.971618 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:29.971618 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:29.971618 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:29.971618 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:29.972523 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:15:29.972523 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:15:29.972523 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:29.972523 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:29.978249 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 348/698] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:15:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:29.981741 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1547824d-4bd8-4e3e-a6c6-bf9f545b0513 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:30.032529 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9fbae9-6862-4f82-8fd1-810006ccd31f None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:30.061801 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:30.098134 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1547824d-4bd8-4e3e-a6c6-bf9f545b0513 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:30.101213 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c700c75-b560-4024-b285-475fbf5f5298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:30.102193 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c700c75-b560-4024-b285-475fbf5f5298 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:30.102688 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c700c75-b560-4024-b285-475fbf5f5298 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:30.103182 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c700c75-b560-4024-b285-475fbf5f5298 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:30.104114 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c700c75-b560-4024-b285-475fbf5f5298 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:30.104624 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c700c75-b560-4024-b285-475fbf5f5298 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:30.106359 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c700c75-b560-4024-b285-475fbf5f5298 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:30.120218 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 351/699] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:15:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:30.125640 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c700c75-b560-4024-b285-475fbf5f5298 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:30.134406 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76ca4c4-67d5-4a31-94e1-03d359f8b9de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:30.135119 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ca4c4-67d5-4a31-94e1-03d359f8b9de None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:30.135399 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ca4c4-67d5-4a31-94e1-03d359f8b9de None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:30.135718 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ca4c4-67d5-4a31-94e1-03d359f8b9de None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:30.136746 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76ca4c4-67d5-4a31-94e1-03d359f8b9de None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:30.136858 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76ca4c4-67d5-4a31-94e1-03d359f8b9de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:30.138258 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76ca4c4-67d5-4a31-94e1-03d359f8b9de None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:30.158235 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 349/700] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:15:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:15:30.159048 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76ca4c4-67d5-4a31-94e1-03d359f8b9de None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:30.215554 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:30.238436 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c700c75-b560-4024-b285-475fbf5f5298 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:30.269509 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76ca4c4-67d5-4a31-94e1-03d359f8b9de None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:30.275612 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7daed3-8b47-4295-b84e-723b31b9c509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:30.276693 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7daed3-8b47-4295-b84e-723b31b9c509 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:30.277179 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7daed3-8b47-4295-b84e-723b31b9c509 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:30.277653 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7daed3-8b47-4295-b84e-723b31b9c509 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:30.286653 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7daed3-8b47-4295-b84e-723b31b9c509 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:30.288380 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7daed3-8b47-4295-b84e-723b31b9c509 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:30.304136 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-9d7daed3-8b47-4295-b84e-723b31b9c509 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 15 19:15:30.304136 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:30.304136 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:30.304136 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:30.304136 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:30.304136 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:30.304136 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:30.304136 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:30.304136 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:30.304136 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:30.304136 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:30.304136 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:30.304136 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 19:15:30.304136 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 19:15:30.304136 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 19:15:30.304136 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 19:15:30.305091 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:30.305091 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:30.305091 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:15:30.305091 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:15:30.305091 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:15:30.305091 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:15:30.305091 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:30.305091 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:30.305091 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:30.305091 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:30.305091 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:30.305091 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:30.305091 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:30.305091 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:30.305091 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:30.305091 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:30.305091 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:30.305091 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:30.306062 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:30.306062 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:30.306062 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 19:15:30.306062 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 19:15:30.306062 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 19:15:30.306062 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 19:15:30.306062 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 19:15:30.306062 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 19:15:30.306062 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 15 19:15:30.306062 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:30.306620 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea152be7-698a-44a2-8047-9ce5f718604f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:30.306710 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea152be7-698a-44a2-8047-9ce5f718604f None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:30.307289 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea152be7-698a-44a2-8047-9ce5f718604f None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:30.307289 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea152be7-698a-44a2-8047-9ce5f718604f None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:30.307994 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea152be7-698a-44a2-8047-9ce5f718604f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:30.309379 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 352/701] 10.222.0.33 () {56 vars in 1093 bytes} [Fri Jul 15 19:15:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:30.309552 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea152be7-698a-44a2-8047-9ce5f718604f None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:30.319335 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7daed3-8b47-4295-b84e-723b31b9c509 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:30.359068 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:30.437707 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 350/702] 10.222.0.33 () {60 vars in 1132 bytes} [Fri Jul 15 19:15:30 2022] POST /identity/v3/users => generated 254 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:15:30.440450 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7daed3-8b47-4295-b84e-723b31b9c509 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:30.497816 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9954f9-a5b4-443f-a9e6-bef2ca254602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:30.498548 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9954f9-a5b4-443f-a9e6-bef2ca254602 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:30.498878 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9954f9-a5b4-443f-a9e6-bef2ca254602 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:30.499159 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9954f9-a5b4-443f-a9e6-bef2ca254602 None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:30.500022 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9954f9-a5b4-443f-a9e6-bef2ca254602 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:30.500335 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9954f9-a5b4-443f-a9e6-bef2ca254602 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:30.501813 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9954f9-a5b4-443f-a9e6-bef2ca254602 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:30.529999 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 353/703] 10.222.0.33 () {56 vars in 1133 bytes} [Fri Jul 15 19:15:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:30.536760 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea152be7-698a-44a2-8047-9ce5f718604f None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:30.609753 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f1462b-6134-41e6-8b78-dc290a90d4b1 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:30.610088 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f1462b-6134-41e6-8b78-dc290a90d4b1 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:30.610332 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f1462b-6134-41e6-8b78-dc290a90d4b1 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:30.611830 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 354/704] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:15:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:15:30.616746 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e9c8fe-f215-4298-98c1-f61c0e6f0c39 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:30.617031 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e9c8fe-f215-4298-98c1-f61c0e6f0c39 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:30.617310 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e9c8fe-f215-4298-98c1-f61c0e6f0c39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:30.647446 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-81e9c8fe-f215-4298-98c1-f61c0e6f0c39 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:15:30.648720 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-81e9c8fe-f215-4298-98c1-f61c0e6f0c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:15:30.666398 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea152be7-698a-44a2-8047-9ce5f718604f None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:30.678553 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:30.712385 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc95aefa-e5bb-47b6-8713-67db5f6c1fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:30.713209 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc95aefa-e5bb-47b6-8713-67db5f6c1fa6 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:30.713621 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc95aefa-e5bb-47b6-8713-67db5f6c1fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:30.713949 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc95aefa-e5bb-47b6-8713-67db5f6c1fa6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:30.730454 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 355/705] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:15:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 19:15:30.753147 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e9c8fe-f215-4298-98c1-f61c0e6f0c39 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:30.778270 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:30.806657 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc95aefa-e5bb-47b6-8713-67db5f6c1fa6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:30.808141 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc95aefa-e5bb-47b6-8713-67db5f6c1fa6 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:30.865371 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e9c8fe-f215-4298-98c1-f61c0e6f0c39 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:30.904030 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd367014-f755-40af-863e-f0d110312159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:30.904850 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd367014-f755-40af-863e-f0d110312159 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:30.905136 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd367014-f755-40af-863e-f0d110312159 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:30.905410 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd367014-f755-40af-863e-f0d110312159 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:30.906267 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd367014-f755-40af-863e-f0d110312159 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:30.914260 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd367014-f755-40af-863e-f0d110312159 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:30.923405 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 351/706] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 15 19:15:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 19:15:30.924649 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc95aefa-e5bb-47b6-8713-67db5f6c1fa6 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:30.933126 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:30.938484 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-bd367014-f755-40af-863e-f0d110312159 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 15 19:15:30.938484 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:30.938484 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 19:15:30.938484 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 19:15:30.938484 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 19:15:30.938484 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 19:15:30.938484 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 19:15:30.938484 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:15:30.938484 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 15 19:15:30.938484 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:30.938484 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 19:15:30.938484 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:30.938484 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:30.938484 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:30.938484 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:30.938484 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:30.939723 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:30.939723 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:30.939723 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:30.939723 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:30.939723 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:30.939723 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:30.939723 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:30.939723 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 19:15:30.939723 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 19:15:30.939723 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 19:15:30.939723 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 19:15:30.939723 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:30.939723 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:30.939723 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:30.939723 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:30.939723 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:30.940731 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:30.940731 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:30.940731 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:30.940731 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:30.940731 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:30.940731 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:30.940731 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:30.940731 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:30.940731 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:30.940731 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:30.940731 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:30.940731 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 19:15:30.940731 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:15:30.940731 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 15 19:15:30.940731 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:30.940731 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 356/707] 10.222.0.33 () {56 vars in 1129 bytes} [Fri Jul 15 19:15:30 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:15:30.945812 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd367014-f755-40af-863e-f0d110312159 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:31.040460 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc95aefa-e5bb-47b6-8713-67db5f6c1fa6 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:31.076297 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61dff14-6a3f-43af-b129-2ab60e06b316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:31.077045 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61dff14-6a3f-43af-b129-2ab60e06b316 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:31.077304 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61dff14-6a3f-43af-b129-2ab60e06b316 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:31.077580 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61dff14-6a3f-43af-b129-2ab60e06b316 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:31.088929 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61dff14-6a3f-43af-b129-2ab60e06b316 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:31.090416 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61dff14-6a3f-43af-b129-2ab60e06b316 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:31.108940 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-f61dff14-6a3f-43af-b129-2ab60e06b316 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:31.108940 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:31.108940 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:31.108940 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:31.108940 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:31.108940 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:31.108940 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:31.108940 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:31.108940 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:31.108940 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:31.108940 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:31.108940 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:31.108940 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:15:31.108940 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:15:31.108940 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:15:31.110132 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:15:31.110132 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:31.110132 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:31.110132 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:31.110132 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:31.110132 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:31.110132 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:31.110132 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:31.110132 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:31.110132 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:31.110132 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:31.110132 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:31.110132 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:31.110132 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:31.110132 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:31.110132 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:31.110132 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:31.111262 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:15:31.111262 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:15:31.111262 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:31.111262 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:31.117799 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 352/708] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:15:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:31.117799 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61dff14-6a3f-43af-b129-2ab60e06b316 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:31.162147 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:31.177914 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:31.184932 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd367014-f755-40af-863e-f0d110312159 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:31.221147 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61dff14-6a3f-43af-b129-2ab60e06b316 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:31.223966 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ddf396-8b8b-41ae-b7de-b1161b6d3976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:31.224967 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ddf396-8b8b-41ae-b7de-b1161b6d3976 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:31.225480 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ddf396-8b8b-41ae-b7de-b1161b6d3976 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:31.225996 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ddf396-8b8b-41ae-b7de-b1161b6d3976 None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:31.226933 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ddf396-8b8b-41ae-b7de-b1161b6d3976 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:31.227468 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ddf396-8b8b-41ae-b7de-b1161b6d3976 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:31.229063 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ddf396-8b8b-41ae-b7de-b1161b6d3976 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:31.234613 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 357/709] 10.222.0.33 () {56 vars in 1124 bytes} [Fri Jul 15 19:15:30 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:15:31.236217 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb1e742-ec26-4b3e-b319-7fc7c014aa60 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:31.236761 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb1e742-ec26-4b3e-b319-7fc7c014aa60 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:31.237269 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb1e742-ec26-4b3e-b319-7fc7c014aa60 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:31.239007 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 358/710] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:15:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:15:31.243615 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb1e742-ec26-4b3e-b319-7fc7c014aa60 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:31.260462 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83525d09-2759-4410-9c0f-39d358615a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:31.261140 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83525d09-2759-4410-9c0f-39d358615a42 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:31.261424 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83525d09-2759-4410-9c0f-39d358615a42 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:31.261701 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83525d09-2759-4410-9c0f-39d358615a42 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:31.262532 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83525d09-2759-4410-9c0f-39d358615a42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:31.262839 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83525d09-2759-4410-9c0f-39d358615a42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:31.264410 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83525d09-2759-4410-9c0f-39d358615a42 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:31.280944 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 353/711] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:15:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:15:31.282370 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816c4088-7202-499d-879c-0bf1d67405e7 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:31.282622 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816c4088-7202-499d-879c-0bf1d67405e7 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:31.282895 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816c4088-7202-499d-879c-0bf1d67405e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:31.301985 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-816c4088-7202-499d-879c-0bf1d67405e7 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:15:31.303214 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-816c4088-7202-499d-879c-0bf1d67405e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:15:31.375405 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb1e742-ec26-4b3e-b319-7fc7c014aa60 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:31.390161 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 354/712] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:15:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:15:31.391465 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816c4088-7202-499d-879c-0bf1d67405e7 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:31.413134 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba0e29c-5bf7-4a86-bcd7-09c7070a229c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:31.413857 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba0e29c-5bf7-4a86-bcd7-09c7070a229c None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:31.414142 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba0e29c-5bf7-4a86-bcd7-09c7070a229c None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:31.414481 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba0e29c-5bf7-4a86-bcd7-09c7070a229c None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:31.415734 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba0e29c-5bf7-4a86-bcd7-09c7070a229c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:31.415734 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba0e29c-5bf7-4a86-bcd7-09c7070a229c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:31.416981 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba0e29c-5bf7-4a86-bcd7-09c7070a229c None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:31.423960 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 359/713] 10.222.0.33 () {56 vars in 1124 bytes} [Fri Jul 15 19:15:31 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:31.439473 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba0e29c-5bf7-4a86-bcd7-09c7070a229c None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:31.462848 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:31.496434 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:31.567111 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816c4088-7202-499d-879c-0bf1d67405e7 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:31.606643 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba0e29c-5bf7-4a86-bcd7-09c7070a229c None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:31.612063 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dab6ce-cc61-4cba-8347-c009bd19b048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:31.612870 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dab6ce-cc61-4cba-8347-c009bd19b048 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:31.613206 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dab6ce-cc61-4cba-8347-c009bd19b048 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:31.613420 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dab6ce-cc61-4cba-8347-c009bd19b048 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:31.623189 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dab6ce-cc61-4cba-8347-c009bd19b048 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:31.624664 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dab6ce-cc61-4cba-8347-c009bd19b048 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:31.639625 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-e1dab6ce-cc61-4cba-8347-c009bd19b048 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:15:31.639625 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:31.639625 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:31.639625 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:31.639625 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:31.639625 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:31.639625 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:31.639625 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:31.639625 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:31.639625 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:31.639625 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:31.639625 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:31.639625 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 19:15:31.639625 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 19:15:31.639625 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 19:15:31.640711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 19:15:31.640711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:31.640711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:31.640711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:31.640711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:31.640711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:31.640711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:31.640711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:31.640711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:31.640711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:31.640711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:31.640711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:31.640711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:31.640711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:31.640711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:31.640711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:31.640711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:31.641612 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 19:15:31.641612 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 19:15:31.641612 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:15:31.641612 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:15:31.641612 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:15:31.641612 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:15:31.641612 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:15:31.641612 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:31.641612 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 355/714] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 15 19:15:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:31.647710 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dab6ce-cc61-4cba-8347-c009bd19b048 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:31.670172 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddb693c-1961-4b2e-8268-aecdebc0a5fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:31.670974 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb693c-1961-4b2e-8268-aecdebc0a5fd None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:31.671255 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb693c-1961-4b2e-8268-aecdebc0a5fd None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:31.671530 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb693c-1961-4b2e-8268-aecdebc0a5fd None admin] PATH_INFO: `/v3/services/network` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:31.672191 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddb693c-1961-4b2e-8268-aecdebc0a5fd None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:31.673575 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddb693c-1961-4b2e-8268-aecdebc0a5fd None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:31.692645 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-cddb693c-1961-4b2e-8268-aecdebc0a5fd None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 15 19:15:31.692645 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:31.692645 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 19:15:31.692645 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 19:15:31.692645 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 19:15:31.692645 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 19:15:31.692645 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 19:15:31.692645 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:15:31.692645 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 15 19:15:31.692645 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:31.692645 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 19:15:31.692645 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:31.692645 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:31.692645 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:31.692645 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:31.692645 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:31.692645 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:31.693977 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:31.693977 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:31.693977 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:31.693977 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:31.693977 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:31.693977 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:31.693977 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 19:15:31.693977 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 19:15:31.693977 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 19:15:31.693977 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 19:15:31.693977 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:31.693977 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:31.693977 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:31.693977 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:31.693977 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:31.693977 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:31.695107 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:31.695107 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:31.695107 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:31.695107 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:31.695107 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:31.695107 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:31.695107 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:31.695107 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:31.695107 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:31.695107 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:31.695107 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 19:15:31.695107 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:15:31.695107 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 15 19:15:31.695107 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:31.695107 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 360/715] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 15 19:15:31 2022] GET /identity/v3/services/network => generated 88 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:31.700261 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddb693c-1961-4b2e-8268-aecdebc0a5fd None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:31.794313 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:31.815178 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddb693c-1961-4b2e-8268-aecdebc0a5fd None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:31.867286 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dab6ce-cc61-4cba-8347-c009bd19b048 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:31.883552 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:31.889536 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac73fe1-29f2-451f-937f-2d58f71d43ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:31.891058 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac73fe1-29f2-451f-937f-2d58f71d43ae None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:31.891510 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac73fe1-29f2-451f-937f-2d58f71d43ae None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:31.891795 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac73fe1-29f2-451f-937f-2d58f71d43ae None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:31.892516 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac73fe1-29f2-451f-937f-2d58f71d43ae None admin] RBAC: Adding query filter params (name=network) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:31.892826 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac73fe1-29f2-451f-937f-2d58f71d43ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:31.894237 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac73fe1-29f2-451f-937f-2d58f71d43ae None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:31.904380 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 361/716] 10.222.0.33 () {56 vars in 1112 bytes} [Fri Jul 15 19:15:31 2022] GET /identity/v3/services?name=network => generated 124 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:15:31.913930 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac73fe1-29f2-451f-937f-2d58f71d43ae None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:31.946116 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c73f783-d8a6-4e9a-820c-7c069f94f1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:31.946834 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c73f783-d8a6-4e9a-820c-7c069f94f1ad None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:31.947123 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c73f783-d8a6-4e9a-820c-7c069f94f1ad None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:31.947396 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c73f783-d8a6-4e9a-820c-7c069f94f1ad None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:31.948191 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c73f783-d8a6-4e9a-820c-7c069f94f1ad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:31.948502 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c73f783-d8a6-4e9a-820c-7c069f94f1ad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:31.949961 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c73f783-d8a6-4e9a-820c-7c069f94f1ad None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:31.978614 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 356/717] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jul 15 19:15:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:31.999425 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c73f783-d8a6-4e9a-820c-7c069f94f1ad None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:32.120916 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac73fe1-29f2-451f-937f-2d58f71d43ae None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:32.173355 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a044763-8408-4914-b02f-561b20c8ec6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:32.174841 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a044763-8408-4914-b02f-561b20c8ec6e None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:32.175328 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a044763-8408-4914-b02f-561b20c8ec6e None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:32.175828 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a044763-8408-4914-b02f-561b20c8ec6e None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:32.176837 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a044763-8408-4914-b02f-561b20c8ec6e None admin] RBAC: Adding query filter params (type=network) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:32.177379 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a044763-8408-4914-b02f-561b20c8ec6e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:32.179123 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a044763-8408-4914-b02f-561b20c8ec6e None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:32.184723 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 362/718] 10.222.0.33 () {56 vars in 1112 bytes} [Fri Jul 15 19:15:31 2022] GET /identity/v3/services?type=network => generated 124 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:32.188223 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c73f783-d8a6-4e9a-820c-7c069f94f1ad None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:32.248408 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517b9ce9-d1b9-4682-bbb0-ff3a672ce9da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:32.249532 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b9ce9-d1b9-4682-bbb0-ff3a672ce9da None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:32.250045 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b9ce9-d1b9-4682-bbb0-ff3a672ce9da None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:32.250592 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b9ce9-d1b9-4682-bbb0-ff3a672ce9da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:32.251681 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517b9ce9-d1b9-4682-bbb0-ff3a672ce9da None admin] RBAC: Adding query filter params (role.id=4318f86ba9f74cd4a0bef1c311276657, scope.project.id=55a756ccaef14cb49b26b0d68dd0b466, user.id=7673a3b5c0324e07a9535ac70ed7a491) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:32.252285 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517b9ce9-d1b9-4682-bbb0-ff3a672ce9da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:32.254142 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517b9ce9-d1b9-4682-bbb0-ff3a672ce9da None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:32.264471 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 357/719] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 15 19:15:31 2022] GET /identity/v3/role_assignments?user.id=7673a3b5c0324e07a9535ac70ed7a491&scope.project.id=55a756ccaef14cb49b26b0d68dd0b466&role.id=4318f86ba9f74cd4a0bef1c311276657 => generated 259 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:15:32.384924 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b1ab5e-57c2-4085-be0e-b850412a86da None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:32.385262 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b1ab5e-57c2-4085-be0e-b850412a86da None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:32.385499 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b1ab5e-57c2-4085-be0e-b850412a86da None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:32.387251 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 363/720] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:15:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:15:32.401514 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c47f6a-0436-4a64-8502-1edbf99af00c None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:32.401876 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c47f6a-0436-4a64-8502-1edbf99af00c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:32.402109 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c47f6a-0436-4a64-8502-1edbf99af00c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:32.431592 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-f5c47f6a-0436-4a64-8502-1edbf99af00c None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:15:32.432735 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-f5c47f6a-0436-4a64-8502-1edbf99af00c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:15:32.470616 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:32.525213 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 358/721] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:15:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:15:32.548428 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b1ab5e-57c2-4085-be0e-b850412a86da None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:32.552950 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:32.676527 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b1ab5e-57c2-4085-be0e-b850412a86da None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:32.719262 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d17c722-4f4f-40c3-b129-9c66d3d619c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:32.719981 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d17c722-4f4f-40c3-b129-9c66d3d619c8 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:32.720266 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d17c722-4f4f-40c3-b129-9c66d3d619c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:32.720543 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d17c722-4f4f-40c3-b129-9c66d3d619c8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:32.741000 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d17c722-4f4f-40c3-b129-9c66d3d619c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:15:32.748910 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-8d17c722-4f4f-40c3-b129-9c66d3d619c8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:15:32.748910 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:32.748910 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:32.748910 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:32.748910 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:32.748910 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:32.748910 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:32.748910 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:32.748910 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:32.748910 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:32.748910 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:32.748910 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:32.748910 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:15:32.748910 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:15:32.748910 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:15:32.748910 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:15:32.749919 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:15:32.749919 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:15:32.749919 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:15:32.749919 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:15:32.749919 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:32.749919 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:32.749919 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:32.749919 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:32.749919 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:32.749919 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:32.749919 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:32.749919 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:32.749919 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:32.749919 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:32.749919 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:32.749919 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:32.750793 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:32.750793 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:32.750793 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:32.750793 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:32.750793 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:15:32.750793 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:15:32.750793 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:15:32.750793 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:15:32.750793 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:15:32.750793 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:15:32.750793 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:15:32.750793 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:32.750793 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 364/722] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jul 15 19:15:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:32.757937 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c47f6a-0436-4a64-8502-1edbf99af00c None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:32.917442 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c47f6a-0436-4a64-8502-1edbf99af00c None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:32.961192 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf2dfab-e388-4427-b603-27a26330d6e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:32.962274 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf2dfab-e388-4427-b603-27a26330d6e9 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:32.962754 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf2dfab-e388-4427-b603-27a26330d6e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:32.963220 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf2dfab-e388-4427-b603-27a26330d6e9 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:32.964186 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf2dfab-e388-4427-b603-27a26330d6e9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:32.964684 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf2dfab-e388-4427-b603-27a26330d6e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:32.966292 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf2dfab-e388-4427-b603-27a26330d6e9 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:32.980592 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 359/723] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 15 19:15:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:32.981540 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d17c722-4f4f-40c3-b129-9c66d3d619c8 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:33.100866 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:33.157908 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d17c722-4f4f-40c3-b129-9c66d3d619c8 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:33.209724 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf9c144-48d9-4351-8738-8f391c5f12f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:33.210552 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf9c144-48d9-4351-8738-8f391c5f12f5 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:33.210839 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf9c144-48d9-4351-8738-8f391c5f12f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:33.211112 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf9c144-48d9-4351-8738-8f391c5f12f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:33.222466 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf9c144-48d9-4351-8738-8f391c5f12f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:33.224027 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf9c144-48d9-4351-8738-8f391c5f12f5 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:33.250308 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-dcf9c144-48d9-4351-8738-8f391c5f12f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:33.250308 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:33.250308 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:33.250308 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:33.250308 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:33.250308 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:33.250308 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:33.250308 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:33.250308 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:33.250308 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:33.250308 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:33.250308 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:33.250308 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:15:33.250308 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:15:33.250308 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:15:33.251303 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:15:33.251303 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:33.251303 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:33.251303 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:33.251303 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:33.251303 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:33.251303 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:33.251303 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:33.251303 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:33.251303 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:33.251303 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:33.251303 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:33.251303 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:33.251303 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:33.251303 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:33.251303 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:33.251303 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:33.252190 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:15:33.252190 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:15:33.252190 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:33.252190 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:33.252190 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 365/724] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:15:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:33.258575 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf9c144-48d9-4351-8738-8f391c5f12f5 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:33.259611 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c0836-3010-462c-a821-ce6ee22a2d14 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:33.260102 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c0836-3010-462c-a821-ce6ee22a2d14 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:33.260575 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c0836-3010-462c-a821-ce6ee22a2d14 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:33.262461 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 360/725] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:15:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:15:33.267795 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cf755c-239d-4f11-b0f2-fc5c470d688d None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:33.268314 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cf755c-239d-4f11-b0f2-fc5c470d688d None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:33.268787 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cf755c-239d-4f11-b0f2-fc5c470d688d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:33.317805 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-e3cf755c-239d-4f11-b0f2-fc5c470d688d None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:15:33.319702 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e3cf755c-239d-4f11-b0f2-fc5c470d688d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:15:33.363990 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:33.401427 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf9c144-48d9-4351-8738-8f391c5f12f5 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:33.430780 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:33.445677 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 361/726] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:15:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 19:15:33.461338 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe44d8d-226d-46b7-8e30-bccb0e8effee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:33.462130 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe44d8d-226d-46b7-8e30-bccb0e8effee None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:33.462479 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe44d8d-226d-46b7-8e30-bccb0e8effee None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:33.462826 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe44d8d-226d-46b7-8e30-bccb0e8effee None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:33.463628 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe44d8d-226d-46b7-8e30-bccb0e8effee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:33.464007 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe44d8d-226d-46b7-8e30-bccb0e8effee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:33.465568 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe44d8d-226d-46b7-8e30-bccb0e8effee None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:33.475456 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cf755c-239d-4f11-b0f2-fc5c470d688d None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:33.492254 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 366/727] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:15:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:33.497275 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe44d8d-226d-46b7-8e30-bccb0e8effee None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:33.594079 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cf755c-239d-4f11-b0f2-fc5c470d688d None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:33.616183 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe44d8d-226d-46b7-8e30-bccb0e8effee None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:33.648026 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3f2edb-4880-4d3b-a40d-4e4abbfc7725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:33.649242 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f2edb-4880-4d3b-a40d-4e4abbfc7725 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:33.649753 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f2edb-4880-4d3b-a40d-4e4abbfc7725 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:33.650262 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f2edb-4880-4d3b-a40d-4e4abbfc7725 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:33.655023 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01af3408-7e9a-4bd0-ba71-c063cdd369b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:33.655836 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01af3408-7e9a-4bd0-ba71-c063cdd369b9 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:33.656120 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01af3408-7e9a-4bd0-ba71-c063cdd369b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:33.656398 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01af3408-7e9a-4bd0-ba71-c063cdd369b9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:33.665004 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3f2edb-4880-4d3b-a40d-4e4abbfc7725 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:15:33.666910 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01af3408-7e9a-4bd0-ba71-c063cdd369b9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:33.668391 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01af3408-7e9a-4bd0-ba71-c063cdd369b9 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:33.674045 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-4f3f2edb-4880-4d3b-a40d-4e4abbfc7725 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:15:33.674045 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:33.674045 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:33.674045 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:33.674045 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:33.674045 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:33.674045 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:33.674045 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:33.674045 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:33.674045 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:33.674045 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:33.674045 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:33.674045 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:15:33.674045 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:15:33.674045 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:15:33.674045 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:15:33.675060 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:15:33.675060 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:15:33.675060 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:15:33.675060 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:15:33.675060 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:33.675060 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:33.675060 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:33.675060 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:33.675060 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:33.675060 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:33.675060 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:33.675060 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:33.675060 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:33.675060 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:33.675060 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:33.675060 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:33.675900 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:33.675900 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:33.675900 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:33.675900 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:33.675900 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:15:33.675900 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:15:33.675900 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:15:33.675900 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:15:33.675900 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:15:33.675900 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:15:33.675900 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:15:33.675900 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:33.678738 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 362/728] 10.222.0.33 () {56 vars in 1102 bytes} [Fri Jul 15 19:15:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:15:33.684736 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3f2edb-4880-4d3b-a40d-4e4abbfc7725 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:33.687118 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-01af3408-7e9a-4bd0-ba71-c063cdd369b9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 15 19:15:33.687118 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:33.687118 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:33.687118 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:33.687118 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:33.687118 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:33.687118 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:33.687118 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:33.687118 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:33.687118 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:33.687118 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:33.687118 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:33.687118 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 19:15:33.687118 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 19:15:33.687118 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 19:15:33.687118 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 19:15:33.688026 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:33.688026 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:33.688026 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:15:33.688026 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:15:33.688026 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:15:33.688026 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:15:33.688026 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:33.688026 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:33.688026 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:33.688026 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:33.688026 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:33.688026 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:33.688026 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:33.688026 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:33.688026 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:33.688026 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:33.688026 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:33.688026 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:33.689019 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:33.689019 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:33.689019 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 19:15:33.689019 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 19:15:33.689019 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 19:15:33.689019 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 19:15:33.689019 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 19:15:33.689019 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 19:15:33.689019 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 15 19:15:33.689019 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:33.689019 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 367/729] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jul 15 19:15:33 2022] GET /identity/v3/users/swift => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:15:33.694251 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01af3408-7e9a-4bd0-ba71-c063cdd369b9 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:33.722995 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:33.734004 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:33.814793 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01af3408-7e9a-4bd0-ba71-c063cdd369b9 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:33.853396 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431b4c98-6119-4856-bd4b-98644097a881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:33.854020 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431b4c98-6119-4856-bd4b-98644097a881 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:33.854306 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431b4c98-6119-4856-bd4b-98644097a881 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:33.854714 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431b4c98-6119-4856-bd4b-98644097a881 None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:33.855563 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431b4c98-6119-4856-bd4b-98644097a881 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:33.855874 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431b4c98-6119-4856-bd4b-98644097a881 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:33.857370 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431b4c98-6119-4856-bd4b-98644097a881 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:33.877875 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 368/730] 10.222.0.33 () {56 vars in 1135 bytes} [Fri Jul 15 19:15:33 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:15:33.885670 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431b4c98-6119-4856-bd4b-98644097a881 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:33.947066 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3f2edb-4880-4d3b-a40d-4e4abbfc7725 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:33.988947 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eadb9ef-305f-43f9-9e27-0ddd8b1000d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:33.990084 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eadb9ef-305f-43f9-9e27-0ddd8b1000d7 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:33.990616 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eadb9ef-305f-43f9-9e27-0ddd8b1000d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:33.991102 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eadb9ef-305f-43f9-9e27-0ddd8b1000d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:33.992043 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eadb9ef-305f-43f9-9e27-0ddd8b1000d7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:33.992551 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eadb9ef-305f-43f9-9e27-0ddd8b1000d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:33.993930 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:33.994364 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eadb9ef-305f-43f9-9e27-0ddd8b1000d7 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:34.001370 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431b4c98-6119-4856-bd4b-98644097a881 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:34.007693 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 363/731] 10.222.0.33 () {56 vars in 1103 bytes} [Fri Jul 15 19:15:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:15:34.009237 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd55e71e-8edd-4456-8071-eb00991d0190 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:34.009686 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd55e71e-8edd-4456-8071-eb00991d0190 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:34.010147 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd55e71e-8edd-4456-8071-eb00991d0190 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:34.012131 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 364/732] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:15:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:15:34.013092 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd55e71e-8edd-4456-8071-eb00991d0190 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:34.039794 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cefecf1-c902-48cd-bbee-d78f6407249c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:34.040451 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cefecf1-c902-48cd-bbee-d78f6407249c None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:34.040734 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cefecf1-c902-48cd-bbee-d78f6407249c None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:34.041026 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cefecf1-c902-48cd-bbee-d78f6407249c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:34.105067 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:34.121223 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cefecf1-c902-48cd-bbee-d78f6407249c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:34.123013 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cefecf1-c902-48cd-bbee-d78f6407249c None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:34.135696 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd55e71e-8edd-4456-8071-eb00991d0190 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:34.173145 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655ee51d-f833-4b7c-9386-9b9488503ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:34.174071 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ee51d-f833-4b7c-9386-9b9488503ab6 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:34.174298 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ee51d-f833-4b7c-9386-9b9488503ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:34.174577 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ee51d-f833-4b7c-9386-9b9488503ab6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:34.186654 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655ee51d-f833-4b7c-9386-9b9488503ab6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:34.188238 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655ee51d-f833-4b7c-9386-9b9488503ab6 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:34.206380 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-655ee51d-f833-4b7c-9386-9b9488503ab6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:34.206380 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:34.206380 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:34.206380 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:34.206380 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:34.206380 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:34.206380 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:34.206380 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:34.206380 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:34.206380 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:34.206380 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:34.206380 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:34.206380 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:15:34.206380 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:15:34.206380 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:15:34.207334 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:15:34.207334 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:34.207334 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:34.207334 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:34.207334 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:34.207334 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:34.207334 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:34.207334 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:34.207334 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:34.207334 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:34.207334 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:34.207334 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:34.207334 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:34.207334 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:34.207334 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:34.207334 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:34.207334 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:34.208894 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:15:34.208894 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:15:34.208894 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:34.208894 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:34.208894 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 365/733] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:15:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:34.209619 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b9c2d0-f011-41f7-9b7f-9ae1252da576 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:34.209724 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b9c2d0-f011-41f7-9b7f-9ae1252da576 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:34.209964 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b9c2d0-f011-41f7-9b7f-9ae1252da576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:34.227365 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-e6b9c2d0-f011-41f7-9b7f-9ae1252da576 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:15:34.228163 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:34.228622 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e6b9c2d0-f011-41f7-9b7f-9ae1252da576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:15:34.236940 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 369/734] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 15 19:15:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 15 19:15:34.238175 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cefecf1-c902-48cd-bbee-d78f6407249c None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:34.311443 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 366/735] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:15:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:15:34.312083 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b9c2d0-f011-41f7-9b7f-9ae1252da576 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:34.342575 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cefecf1-c902-48cd-bbee-d78f6407249c None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:34.379167 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c70f2b-92ff-4d4b-9ed7-84e647f4317c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:34.380354 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c70f2b-92ff-4d4b-9ed7-84e647f4317c None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:34.380825 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c70f2b-92ff-4d4b-9ed7-84e647f4317c None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:34.381259 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c70f2b-92ff-4d4b-9ed7-84e647f4317c None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:34.382428 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c70f2b-92ff-4d4b-9ed7-84e647f4317c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:34.382826 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c70f2b-92ff-4d4b-9ed7-84e647f4317c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:34.384540 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c70f2b-92ff-4d4b-9ed7-84e647f4317c None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:34.397388 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 370/736] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:15:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:34.397985 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c70f2b-92ff-4d4b-9ed7-84e647f4317c None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:34.422985 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:34.456848 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:34.480432 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b9c2d0-f011-41f7-9b7f-9ae1252da576 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:34.505479 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c70f2b-92ff-4d4b-9ed7-84e647f4317c None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:34.526475 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a01c13c-dae8-4ac7-ae17-da6eb4812e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:34.527896 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a01c13c-dae8-4ac7-ae17-da6eb4812e6d None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:34.528587 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a01c13c-dae8-4ac7-ae17-da6eb4812e6d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:34.529078 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a01c13c-dae8-4ac7-ae17-da6eb4812e6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:34.540941 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be87ac3-4b6a-4dc4-812a-0f8677ae2a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:34.541996 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be87ac3-4b6a-4dc4-812a-0f8677ae2a48 None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:34.541996 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be87ac3-4b6a-4dc4-812a-0f8677ae2a48 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:34.542327 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be87ac3-4b6a-4dc4-812a-0f8677ae2a48 None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:34.542998 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be87ac3-4b6a-4dc4-812a-0f8677ae2a48 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:34.544406 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be87ac3-4b6a-4dc4-812a-0f8677ae2a48 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:34.544853 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a01c13c-dae8-4ac7-ae17-da6eb4812e6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:34.546738 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a01c13c-dae8-4ac7-ae17-da6eb4812e6d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:34.566255 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-7a01c13c-dae8-4ac7-ae17-da6eb4812e6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:34.566255 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:34.566255 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:34.566255 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:34.566255 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:34.566255 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:34.566255 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:34.566255 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:34.566255 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:34.566255 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:34.566255 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:34.566255 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:34.566255 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:15:34.566255 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:15:34.566255 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:15:34.567881 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:15:34.567881 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:34.567881 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:34.567881 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:34.567881 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:34.567881 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:34.567881 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:34.567881 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:34.567881 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:34.567881 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:34.567881 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:34.567881 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:34.567881 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:34.567881 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:34.567881 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:34.567881 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:34.567881 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:34.568811 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:15:34.568811 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:15:34.568811 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:34.568811 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:34.568811 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 367/737] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:15:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:34.572223 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a01c13c-dae8-4ac7-ae17-da6eb4812e6d None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:34.641881 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 371/738] 10.222.0.33 () {60 vars in 1141 bytes} [Fri Jul 15 19:15:34 2022] POST /identity/v3/services => generated 251 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:15:34.641881 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be87ac3-4b6a-4dc4-812a-0f8677ae2a48 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:34.695414 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a01c13c-dae8-4ac7-ae17-da6eb4812e6d None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:34.713780 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:34.752708 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51ae44b-c7d0-407b-a579-0fb1b67a16f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:34.753539 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ae44b-c7d0-407b-a579-0fb1b67a16f6 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:34.753847 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ae44b-c7d0-407b-a579-0fb1b67a16f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:34.754126 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ae44b-c7d0-407b-a579-0fb1b67a16f6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:34.763173 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51ae44b-c7d0-407b-a579-0fb1b67a16f6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:34.764663 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51ae44b-c7d0-407b-a579-0fb1b67a16f6 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:34.765115 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be87ac3-4b6a-4dc4-812a-0f8677ae2a48 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:34.785545 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-b51ae44b-c7d0-407b-a579-0fb1b67a16f6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 15 19:15:34.785545 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:34.785545 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:34.785545 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:34.785545 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:34.785545 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:34.785545 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:34.785545 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:34.785545 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:34.785545 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:34.785545 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:34.785545 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:34.785545 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 19:15:34.785545 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 19:15:34.785545 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 19:15:34.785545 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 19:15:34.786676 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:34.786676 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:34.786676 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:15:34.786676 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:15:34.786676 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:15:34.786676 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:15:34.786676 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:34.786676 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:34.786676 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:34.786676 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:34.786676 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:34.786676 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:34.786676 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:34.786676 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:34.786676 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:34.786676 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:34.786676 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:34.786676 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:34.787616 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:34.787616 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:34.787616 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 19:15:34.787616 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 19:15:34.787616 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 19:15:34.787616 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 19:15:34.787616 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 19:15:34.787616 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 19:15:34.787616 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 15 19:15:34.787616 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:34.788176 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:34.791476 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 368/739] 10.222.0.33 () {56 vars in 1117 bytes} [Fri Jul 15 19:15:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:34.793131 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbdfb85-fa79-4735-a615-d272c69260d0 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:34.808101 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbdfb85-fa79-4735-a615-d272c69260d0 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:34.808101 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbdfb85-fa79-4735-a615-d272c69260d0 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:34.808101 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 369/740] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:15:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:15:34.808101 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665c7d24-933f-440a-866b-fcbe79a2f74d None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:34.808101 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665c7d24-933f-440a-866b-fcbe79a2f74d None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:34.808101 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665c7d24-933f-440a-866b-fcbe79a2f74d None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:34.808101 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 370/741] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:15:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:15:34.808726 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665c7d24-933f-440a-866b-fcbe79a2f74d None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:34.816026 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003af4ca-bb7e-4536-abc1-18bacd7146e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:34.816797 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003af4ca-bb7e-4536-abc1-18bacd7146e5 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:34.817081 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003af4ca-bb7e-4536-abc1-18bacd7146e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:34.817503 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003af4ca-bb7e-4536-abc1-18bacd7146e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:34.818311 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003af4ca-bb7e-4536-abc1-18bacd7146e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:34.818617 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003af4ca-bb7e-4536-abc1-18bacd7146e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:34.820002 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003af4ca-bb7e-4536-abc1-18bacd7146e5 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:34.832390 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 372/742] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:15:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:34.834279 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374eaac5-945a-41a7-aa00-2bc506e8130d None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:34.834562 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374eaac5-945a-41a7-aa00-2bc506e8130d None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:34.835744 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374eaac5-945a-41a7-aa00-2bc506e8130d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:34.853570 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-374eaac5-945a-41a7-aa00-2bc506e8130d None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:15:34.854870 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-374eaac5-945a-41a7-aa00-2bc506e8130d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:15:34.919305 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665c7d24-933f-440a-866b-fcbe79a2f74d None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:34.927481 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 373/743] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:15:34 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:15:34.928866 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bacee6-00ae-419e-a10f-7c828439b91d None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:34.929162 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bacee6-00ae-419e-a10f-7c828439b91d None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:34.929440 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bacee6-00ae-419e-a10f-7c828439b91d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:34.947597 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-53bacee6-00ae-419e-a10f-7c828439b91d None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:15:34.948900 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-53bacee6-00ae-419e-a10f-7c828439b91d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:15:34.967353 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf93117e-b503-4d33-87a9-3f7f776fa38b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:34.968026 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf93117e-b503-4d33-87a9-3f7f776fa38b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:34.968356 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf93117e-b503-4d33-87a9-3f7f776fa38b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:34.968620 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf93117e-b503-4d33-87a9-3f7f776fa38b None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:34.969455 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf93117e-b503-4d33-87a9-3f7f776fa38b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:34.969845 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf93117e-b503-4d33-87a9-3f7f776fa38b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:34.971487 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf93117e-b503-4d33-87a9-3f7f776fa38b None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:34.995160 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 371/744] 10.222.0.33 () {56 vars in 1149 bytes} [Fri Jul 15 19:15:34 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:34.995160 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf93117e-b503-4d33-87a9-3f7f776fa38b None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:35.018436 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:35.033059 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 374/745] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:15:34 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:15:35.039214 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bacee6-00ae-419e-a10f-7c828439b91d None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:35.074655 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:35.109796 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf93117e-b503-4d33-87a9-3f7f776fa38b None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:35.142747 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bacee6-00ae-419e-a10f-7c828439b91d None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:35.147201 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6829036b-3651-4d5a-b9e7-0c1e1b20948f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:35.148174 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6829036b-3651-4d5a-b9e7-0c1e1b20948f None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:35.148597 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6829036b-3651-4d5a-b9e7-0c1e1b20948f None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:35.149047 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6829036b-3651-4d5a-b9e7-0c1e1b20948f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:35.156277 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6829036b-3651-4d5a-b9e7-0c1e1b20948f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:35.157900 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6829036b-3651-4d5a-b9e7-0c1e1b20948f None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:35.171687 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-6829036b-3651-4d5a-b9e7-0c1e1b20948f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:15:35.171687 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:35.171687 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:35.171687 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:35.171687 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:35.171687 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:35.171687 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:35.171687 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:35.171687 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:35.171687 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:35.171687 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:35.171687 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:35.171687 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 19:15:35.171687 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 19:15:35.171687 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 19:15:35.172759 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 19:15:35.172759 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:35.172759 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:35.172759 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:35.172759 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:35.172759 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:35.172759 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:35.172759 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:35.172759 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:35.172759 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:35.172759 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:35.172759 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:35.172759 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:35.172759 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:35.172759 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:35.172759 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:35.172759 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:35.173876 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 19:15:35.173876 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 19:15:35.173876 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:15:35.173876 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:15:35.173876 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:15:35.173876 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:15:35.173876 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:15:35.173876 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:35.175637 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 372/746] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 15 19:15:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:35.176690 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6829036b-3651-4d5a-b9e7-0c1e1b20948f None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:35.185115 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e80a3c-5e87-40a9-b1d8-b4a7b3915441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:35.185949 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e80a3c-5e87-40a9-b1d8-b4a7b3915441 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:35.186235 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e80a3c-5e87-40a9-b1d8-b4a7b3915441 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:35.186510 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e80a3c-5e87-40a9-b1d8-b4a7b3915441 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:35.197700 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e80a3c-5e87-40a9-b1d8-b4a7b3915441 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:15:35.205154 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-26e80a3c-5e87-40a9-b1d8-b4a7b3915441 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:15:35.205154 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:35.205154 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:35.205154 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:35.205154 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:35.205154 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:35.205154 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:35.205154 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:35.205154 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:35.205154 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:35.205154 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:35.205154 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:35.205154 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:15:35.205154 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:15:35.205154 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:15:35.205154 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:15:35.206108 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:15:35.206108 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:15:35.206108 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:15:35.206108 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:15:35.206108 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:35.206108 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:35.206108 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:35.206108 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:35.206108 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:35.206108 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:35.206108 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:35.206108 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:35.206108 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:35.206108 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:35.206108 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:35.206108 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:35.206945 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:35.206945 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:35.206945 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:35.206945 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:35.206945 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:15:35.206945 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:15:35.206945 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:15:35.206945 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:15:35.206945 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:15:35.206945 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:15:35.206945 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:15:35.206945 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:35.206945 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 375/747] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jul 15 19:15:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:35.207781 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e80a3c-5e87-40a9-b1d8-b4a7b3915441 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:35.275901 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:35.292372 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6829036b-3651-4d5a-b9e7-0c1e1b20948f None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:35.316111 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:35.323120 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e80a3c-5e87-40a9-b1d8-b4a7b3915441 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:35.329640 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6b72f4-54f8-4f4b-8294-f9aa2cbf32c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:35.330396 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6b72f4-54f8-4f4b-8294-f9aa2cbf32c1 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:35.330818 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6b72f4-54f8-4f4b-8294-f9aa2cbf32c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:35.330944 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6b72f4-54f8-4f4b-8294-f9aa2cbf32c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:35.358032 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56e1e2d-cb2f-4e42-b901-16d33a7ad560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:35.358690 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e1e2d-cb2f-4e42-b901-16d33a7ad560 None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:35.358970 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e1e2d-cb2f-4e42-b901-16d33a7ad560 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:35.359248 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e1e2d-cb2f-4e42-b901-16d33a7ad560 None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:35.359963 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56e1e2d-cb2f-4e42-b901-16d33a7ad560 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:35.361352 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56e1e2d-cb2f-4e42-b901-16d33a7ad560 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:35.404793 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6b72f4-54f8-4f4b-8294-f9aa2cbf32c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:35.406294 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6b72f4-54f8-4f4b-8294-f9aa2cbf32c1 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:35.454759 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 376/748] 10.222.0.33 () {60 vars in 1141 bytes} [Fri Jul 15 19:15:35 2022] POST /identity/v3/services => generated 240 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:15:35.455445 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56e1e2d-cb2f-4e42-b901-16d33a7ad560 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:35.511371 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:35.546263 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 373/749] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 15 19:15:35 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 19:15:35.548161 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6b72f4-54f8-4f4b-8294-f9aa2cbf32c1 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:35.610325 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56e1e2d-cb2f-4e42-b901-16d33a7ad560 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:35.664020 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c8ad90-51a6-421c-90da-9f3d44261a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:35.664693 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8ad90-51a6-421c-90da-9f3d44261a3b None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:35.664979 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8ad90-51a6-421c-90da-9f3d44261a3b None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:35.665308 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8ad90-51a6-421c-90da-9f3d44261a3b None admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:35.666138 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c8ad90-51a6-421c-90da-9f3d44261a3b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:35.666447 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c8ad90-51a6-421c-90da-9f3d44261a3b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:35.667883 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c8ad90-51a6-421c-90da-9f3d44261a3b None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:35.682530 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 377/750] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jul 15 19:15:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:35.682704 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6b72f4-54f8-4f4b-8294-f9aa2cbf32c1 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:35.683756 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c8ad90-51a6-421c-90da-9f3d44261a3b None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:35.699909 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:35.724240 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c6a603-1aa1-4aeb-b320-d4c849c81f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:35.724878 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c6a603-1aa1-4aeb-b320-d4c849c81f70 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:35.725145 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c6a603-1aa1-4aeb-b320-d4c849c81f70 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:35.725419 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c6a603-1aa1-4aeb-b320-d4c849c81f70 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:35.726221 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c6a603-1aa1-4aeb-b320-d4c849c81f70 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:35.726659 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c6a603-1aa1-4aeb-b320-d4c849c81f70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:35.728388 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c6a603-1aa1-4aeb-b320-d4c849c81f70 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:35.741853 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 374/751] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 15 19:15:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:35.743028 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c6a603-1aa1-4aeb-b320-d4c849c81f70 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:35.815339 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:35.844074 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c8ad90-51a6-421c-90da-9f3d44261a3b None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:35.888843 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c6a603-1aa1-4aeb-b320-d4c849c81f70 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:35.893187 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3380b31-a619-403c-aea5-7eaaa2587b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:35.894176 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3380b31-a619-403c-aea5-7eaaa2587b71 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:35.894714 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3380b31-a619-403c-aea5-7eaaa2587b71 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:35.895251 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3380b31-a619-403c-aea5-7eaaa2587b71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:35.920460 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3380b31-a619-403c-aea5-7eaaa2587b71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:35.922903 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3380b31-a619-403c-aea5-7eaaa2587b71 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:35.934394 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5366250-1178-45be-9f87-9b980b5fc184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:35.935203 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5366250-1178-45be-9f87-9b980b5fc184 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:35.935646 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5366250-1178-45be-9f87-9b980b5fc184 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:35.936016 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5366250-1178-45be-9f87-9b980b5fc184 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:35.937538 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5366250-1178-45be-9f87-9b980b5fc184 None admin] RBAC: Adding query filter params (role.id=4318f86ba9f74cd4a0bef1c311276657, scope.project.id=55a756ccaef14cb49b26b0d68dd0b466, user.id=ef76f7fe39a3499aaeebf85c176804f3) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:35.937978 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5366250-1178-45be-9f87-9b980b5fc184 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:35.939809 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5366250-1178-45be-9f87-9b980b5fc184 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:35.946154 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 375/752] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 15 19:15:35 2022] GET /identity/v3/role_assignments?user.id=ef76f7fe39a3499aaeebf85c176804f3&scope.project.id=55a756ccaef14cb49b26b0d68dd0b466&role.id=4318f86ba9f74cd4a0bef1c311276657 => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:35.947101 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5366250-1178-45be-9f87-9b980b5fc184 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:35.948352 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-f3380b31-a619-403c-aea5-7eaaa2587b71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:35.948352 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:35.948352 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:35.948352 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:35.948352 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:35.948352 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:35.948352 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:35.948352 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:35.948352 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:35.948352 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:35.948352 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:35.948352 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:35.948352 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:15:35.948352 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:15:35.948352 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:15:35.949304 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:15:35.949304 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:35.949304 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:35.949304 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:35.949304 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:35.949304 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:35.949304 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:35.949304 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:35.949304 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:35.949304 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:35.949304 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:35.949304 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:35.949304 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:35.949304 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:35.949304 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:35.949304 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:35.949304 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:35.950243 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:15:35.950243 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:15:35.950243 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:35.950243 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:35.958837 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 378/753] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:15:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:35.961565 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3380b31-a619-403c-aea5-7eaaa2587b71 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:36.096998 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:36.137460 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5366250-1178-45be-9f87-9b980b5fc184 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:36.151605 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:36.171470 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3380b31-a619-403c-aea5-7eaaa2587b71 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:36.182693 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cfbbbd-ab16-4477-ae09-37bff281a2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:36.184499 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cfbbbd-ab16-4477-ae09-37bff281a2f4 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:36.185363 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cfbbbd-ab16-4477-ae09-37bff281a2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:36.185863 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cfbbbd-ab16-4477-ae09-37bff281a2f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:36.206957 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cfbbbd-ab16-4477-ae09-37bff281a2f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:36.209324 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cfbbbd-ab16-4477-ae09-37bff281a2f4 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:36.229602 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-a8cfbbbd-ab16-4477-ae09-37bff281a2f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:36.229602 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:36.229602 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:36.229602 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:36.229602 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:36.229602 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:36.229602 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:36.229602 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:36.229602 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:36.229602 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:36.229602 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:36.229602 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:36.229602 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:15:36.229602 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:15:36.229602 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:15:36.230590 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:15:36.230590 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:36.230590 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:36.230590 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:36.230590 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:36.230590 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:36.230590 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:36.230590 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:36.230590 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:36.230590 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:36.230590 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:36.230590 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:36.230590 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:36.230590 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:36.230590 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:36.230590 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:36.230590 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:36.231493 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:15:36.231493 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:15:36.231493 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:36.231493 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:36.233736 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 376/754] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:15:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:36.242696 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cfbbbd-ab16-4477-ae09-37bff281a2f4 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:36.249579 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30357551-962b-4468-b460-9b85f5f3bb24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:36.250302 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30357551-962b-4468-b460-9b85f5f3bb24 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:36.250588 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30357551-962b-4468-b460-9b85f5f3bb24 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:36.250869 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30357551-962b-4468-b460-9b85f5f3bb24 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:36.251604 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30357551-962b-4468-b460-9b85f5f3bb24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:36.251913 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30357551-962b-4468-b460-9b85f5f3bb24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:36.253298 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30357551-962b-4468-b460-9b85f5f3bb24 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:36.280380 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 379/755] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:15:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:15:36.287261 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30357551-962b-4468-b460-9b85f5f3bb24 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:36.358158 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cfbbbd-ab16-4477-ae09-37bff281a2f4 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:36.394620 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765d8705-8fbd-4939-937a-2b1ca32dfbd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:36.396176 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765d8705-8fbd-4939-937a-2b1ca32dfbd2 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:36.396924 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765d8705-8fbd-4939-937a-2b1ca32dfbd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:36.397487 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765d8705-8fbd-4939-937a-2b1ca32dfbd2 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:36.398452 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765d8705-8fbd-4939-937a-2b1ca32dfbd2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:36.398977 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765d8705-8fbd-4939-937a-2b1ca32dfbd2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:36.400579 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765d8705-8fbd-4939-937a-2b1ca32dfbd2 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:36.415647 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 377/756] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:15:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:36.421233 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765d8705-8fbd-4939-937a-2b1ca32dfbd2 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:36.463941 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:36.502152 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30357551-962b-4468-b460-9b85f5f3bb24 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:36.539743 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:36.551949 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552d8db9-b308-47f0-95b7-09cf2d8387f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:36.552853 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552d8db9-b308-47f0-95b7-09cf2d8387f8 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:36.553144 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552d8db9-b308-47f0-95b7-09cf2d8387f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:36.553417 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552d8db9-b308-47f0-95b7-09cf2d8387f8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:36.569318 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552d8db9-b308-47f0-95b7-09cf2d8387f8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:36.570783 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552d8db9-b308-47f0-95b7-09cf2d8387f8 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:36.591324 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-552d8db9-b308-47f0-95b7-09cf2d8387f8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:15:36.591324 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:36.591324 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:36.591324 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:36.591324 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:36.591324 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:36.591324 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:36.591324 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:36.591324 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:36.591324 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:36.591324 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:36.591324 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:36.591324 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 19:15:36.591324 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 19:15:36.591324 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 19:15:36.592367 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 19:15:36.592367 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:36.592367 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:36.592367 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:36.592367 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:36.592367 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:36.592367 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:36.592367 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:36.592367 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:36.592367 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:36.592367 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:36.592367 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:36.592367 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:36.592367 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:36.592367 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:36.592367 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:36.592367 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:36.593285 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 19:15:36.593285 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 19:15:36.593285 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:15:36.593285 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:15:36.593285 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:15:36.593285 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:15:36.593285 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:15:36.593285 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:36.593285 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 380/757] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 15 19:15:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:15:36.602671 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552d8db9-b308-47f0-95b7-09cf2d8387f8 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:36.607608 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765d8705-8fbd-4939-937a-2b1ca32dfbd2 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:36.644022 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de4164a-1cfb-486c-a42d-1cf158bcd46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:36.644837 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de4164a-1cfb-486c-a42d-1cf158bcd46d None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:36.645118 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de4164a-1cfb-486c-a42d-1cf158bcd46d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:36.645454 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de4164a-1cfb-486c-a42d-1cf158bcd46d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:36.653018 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de4164a-1cfb-486c-a42d-1cf158bcd46d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:36.654504 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de4164a-1cfb-486c-a42d-1cf158bcd46d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:36.677305 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-8de4164a-1cfb-486c-a42d-1cf158bcd46d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 15 19:15:36.677305 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:36.677305 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:36.677305 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:36.677305 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:36.677305 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:36.677305 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:36.677305 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:36.677305 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:36.677305 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:36.677305 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:36.677305 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:36.677305 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 19:15:36.677305 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 19:15:36.677305 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 19:15:36.677305 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 19:15:36.678388 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:36.678388 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:36.678388 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:15:36.678388 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:15:36.678388 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:15:36.678388 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:15:36.678388 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:36.678388 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:36.678388 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:36.678388 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:36.678388 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:36.678388 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:36.678388 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:36.678388 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:36.678388 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:36.678388 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:36.678388 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:36.678388 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:36.679372 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:36.679372 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:36.679372 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 19:15:36.679372 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 19:15:36.679372 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 19:15:36.679372 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 19:15:36.679372 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 19:15:36.679372 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 19:15:36.679372 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 15 19:15:36.679372 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:36.679372 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 378/758] 10.222.0.33 () {56 vars in 1093 bytes} [Fri Jul 15 19:15:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:15:36.684193 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de4164a-1cfb-486c-a42d-1cf158bcd46d None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:36.792485 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de4164a-1cfb-486c-a42d-1cf158bcd46d None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:36.823459 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:36.834707 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f889d645-1289-4ad7-96ef-cdec3be0ce1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:36.835490 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f889d645-1289-4ad7-96ef-cdec3be0ce1a None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:36.835748 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f889d645-1289-4ad7-96ef-cdec3be0ce1a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:36.836020 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f889d645-1289-4ad7-96ef-cdec3be0ce1a None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:36.836861 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f889d645-1289-4ad7-96ef-cdec3be0ce1a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:36.837164 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f889d645-1289-4ad7-96ef-cdec3be0ce1a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:36.838634 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f889d645-1289-4ad7-96ef-cdec3be0ce1a None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:36.851536 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552d8db9-b308-47f0-95b7-09cf2d8387f8 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:36.858886 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 379/759] 10.222.0.33 () {56 vars in 1133 bytes} [Fri Jul 15 19:15:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:15:36.860304 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2f840-a5c0-41e3-8d96-fbaf2bf4c23b None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:36.860591 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2f840-a5c0-41e3-8d96-fbaf2bf4c23b None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:36.860864 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2f840-a5c0-41e3-8d96-fbaf2bf4c23b None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:36.866607 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 380/760] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:15:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:15:36.867454 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef2f840-a5c0-41e3-8d96-fbaf2bf4c23b None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:36.898185 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ba80b1-4a0d-49c1-a33a-5076fe76fba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:36.898918 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ba80b1-4a0d-49c1-a33a-5076fe76fba4 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:36.899199 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ba80b1-4a0d-49c1-a33a-5076fe76fba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:36.899777 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ba80b1-4a0d-49c1-a33a-5076fe76fba4 None admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:36.900731 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ba80b1-4a0d-49c1-a33a-5076fe76fba4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:36.901137 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ba80b1-4a0d-49c1-a33a-5076fe76fba4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:36.902791 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ba80b1-4a0d-49c1-a33a-5076fe76fba4 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:36.922525 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 381/761] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jul 15 19:15:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:36.927633 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2ef97e-c00a-4f84-be81-5138aff1e7be None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:36.928140 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2ef97e-c00a-4f84-be81-5138aff1e7be None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:36.929589 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2ef97e-c00a-4f84-be81-5138aff1e7be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:36.953188 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-9b2ef97e-c00a-4f84-be81-5138aff1e7be None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:15:36.955108 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-9b2ef97e-c00a-4f84-be81-5138aff1e7be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:15:36.993949 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:37.001792 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef2f840-a5c0-41e3-8d96-fbaf2bf4c23b None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:37.028657 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 382/762] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:15:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:15:37.029872 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2ef97e-c00a-4f84-be81-5138aff1e7be None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:37.063850 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20040d8f-4190-4397-810b-e44fc370cccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:37.064713 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20040d8f-4190-4397-810b-e44fc370cccb None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:37.065062 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20040d8f-4190-4397-810b-e44fc370cccb None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:37.065405 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20040d8f-4190-4397-810b-e44fc370cccb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:37.075050 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:37.077361 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20040d8f-4190-4397-810b-e44fc370cccb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:37.079055 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20040d8f-4190-4397-810b-e44fc370cccb None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:37.098954 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-20040d8f-4190-4397-810b-e44fc370cccb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:37.098954 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:37.098954 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:37.098954 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:37.098954 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:37.098954 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:37.098954 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:37.098954 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:37.098954 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:37.098954 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:37.098954 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:37.098954 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:37.098954 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:15:37.098954 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:15:37.098954 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:15:37.100071 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:15:37.100071 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:37.100071 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:37.100071 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:37.100071 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:37.100071 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:37.100071 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:37.100071 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:37.100071 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:37.100071 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:37.100071 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:37.100071 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:37.100071 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:37.100071 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:37.100071 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:37.100071 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:37.100071 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:37.101176 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:15:37.101176 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:15:37.101176 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:37.101176 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:37.107141 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 381/763] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:15:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:37.108089 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20040d8f-4190-4397-810b-e44fc370cccb None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:37.153383 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2ef97e-c00a-4f84-be81-5138aff1e7be None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:37.191129 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f75ab66-6b50-49e9-8bc8-cec384ae5841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:37.192122 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f75ab66-6b50-49e9-8bc8-cec384ae5841 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:37.192581 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f75ab66-6b50-49e9-8bc8-cec384ae5841 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:37.193010 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f75ab66-6b50-49e9-8bc8-cec384ae5841 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:37.194124 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f75ab66-6b50-49e9-8bc8-cec384ae5841 None admin] RBAC: Adding query filter params (role.id=70de044d6962417593b6d03571e7502e, scope.project.id=55a756ccaef14cb49b26b0d68dd0b466, user.id=3d893c14620b4a129842062f14a68a78) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:37.194674 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f75ab66-6b50-49e9-8bc8-cec384ae5841 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:37.196279 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f75ab66-6b50-49e9-8bc8-cec384ae5841 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:37.209914 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 383/764] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 15 19:15:37 2022] GET /identity/v3/role_assignments?user.id=3d893c14620b4a129842062f14a68a78&scope.project.id=55a756ccaef14cb49b26b0d68dd0b466&role.id=70de044d6962417593b6d03571e7502e => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:37.211309 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f75ab66-6b50-49e9-8bc8-cec384ae5841 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:37.214748 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20040d8f-4190-4397-810b-e44fc370cccb None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:37.254422 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36a47f0-ba80-4199-8b3f-c6c59acc0825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:37.255329 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36a47f0-ba80-4199-8b3f-c6c59acc0825 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:37.255614 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36a47f0-ba80-4199-8b3f-c6c59acc0825 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:37.255888 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36a47f0-ba80-4199-8b3f-c6c59acc0825 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:37.256592 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36a47f0-ba80-4199-8b3f-c6c59acc0825 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:37.258808 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36a47f0-ba80-4199-8b3f-c6c59acc0825 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:37.292147 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-e36a47f0-ba80-4199-8b3f-c6c59acc0825 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 15 19:15:37.292147 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:37.292147 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 19:15:37.292147 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 19:15:37.292147 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 19:15:37.292147 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 19:15:37.292147 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 19:15:37.292147 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:15:37.292147 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 15 19:15:37.292147 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:37.292147 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 19:15:37.292147 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:37.292147 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:37.292147 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:37.292147 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:37.292147 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:37.293161 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:37.293161 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:37.293161 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:37.293161 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:37.293161 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:37.293161 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:37.293161 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:37.293161 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 19:15:37.293161 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 19:15:37.293161 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 19:15:37.293161 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 19:15:37.293161 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:37.293161 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:37.293161 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:37.293161 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:37.293161 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:37.294046 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:37.294046 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:37.294046 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:37.294046 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:37.294046 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:37.294046 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:37.294046 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:37.294046 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:37.294046 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:37.294046 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:37.294046 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:37.294046 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 19:15:37.294046 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:15:37.294046 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 15 19:15:37.294046 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:37.302524 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 382/765] 10.222.0.33 () {56 vars in 1129 bytes} [Fri Jul 15 19:15:37 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:37.308976 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:37.309943 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36a47f0-ba80-4199-8b3f-c6c59acc0825 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:37.404532 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:37.433209 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f75ab66-6b50-49e9-8bc8-cec384ae5841 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:37.453081 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36a47f0-ba80-4199-8b3f-c6c59acc0825 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:37.472908 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5395cd2-fdb8-4221-a813-2e8726c27efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:37.474431 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5395cd2-fdb8-4221-a813-2e8726c27efc None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:37.475117 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5395cd2-fdb8-4221-a813-2e8726c27efc None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:37.475609 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5395cd2-fdb8-4221-a813-2e8726c27efc None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:37.476571 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5395cd2-fdb8-4221-a813-2e8726c27efc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:37.477071 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5395cd2-fdb8-4221-a813-2e8726c27efc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:37.478850 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5395cd2-fdb8-4221-a813-2e8726c27efc None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:37.486575 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b22a75-d70f-4fad-bda4-4814bc667d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:37.487239 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b22a75-d70f-4fad-bda4-4814bc667d96 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:37.487518 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b22a75-d70f-4fad-bda4-4814bc667d96 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:37.487790 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b22a75-d70f-4fad-bda4-4814bc667d96 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:37.488500 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b22a75-d70f-4fad-bda4-4814bc667d96 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:37.488803 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b22a75-d70f-4fad-bda4-4814bc667d96 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:37.490851 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b22a75-d70f-4fad-bda4-4814bc667d96 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:37.495906 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 384/767] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:15:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:37.500391 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 383/767] 10.222.0.33 () {56 vars in 1124 bytes} [Fri Jul 15 19:15:37 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:37.501615 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5395cd2-fdb8-4221-a813-2e8726c27efc None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:37.501745 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b22a75-d70f-4fad-bda4-4814bc667d96 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:37.605034 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:37.612321 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5395cd2-fdb8-4221-a813-2e8726c27efc None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:37.649080 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c86e8b6-4b18-4867-86ea-98bec41fff59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:37.650218 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c86e8b6-4b18-4867-86ea-98bec41fff59 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:37.650683 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c86e8b6-4b18-4867-86ea-98bec41fff59 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:37.651164 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c86e8b6-4b18-4867-86ea-98bec41fff59 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:37.658869 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c86e8b6-4b18-4867-86ea-98bec41fff59 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:37.660802 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c86e8b6-4b18-4867-86ea-98bec41fff59 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:37.682354 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-1c86e8b6-4b18-4867-86ea-98bec41fff59 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:15:37.682354 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:37.682354 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:37.682354 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:37.682354 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:37.682354 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:37.682354 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:37.682354 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:37.682354 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:37.682354 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:37.682354 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:37.682354 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:37.682354 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 19:15:37.682354 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 19:15:37.682354 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 19:15:37.683313 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 19:15:37.683313 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:37.683313 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:37.683313 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:37.683313 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:37.683313 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:37.683313 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:37.683313 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:37.683313 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:37.683313 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:37.683313 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:37.683313 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:37.683313 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:37.683313 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:37.683313 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:37.683313 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:37.683313 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:37.684201 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 19:15:37.684201 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 19:15:37.684201 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:15:37.684201 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:15:37.684201 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:15:37.684201 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:15:37.684201 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:15:37.684201 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:37.689654 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 385/768] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 15 19:15:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:15:37.692223 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1bc5f4-e3fa-4fb3-9d74-1da50331bc5a None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:37.695628 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b22a75-d70f-4fad-bda4-4814bc667d96 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:37.697461 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1bc5f4-e3fa-4fb3-9d74-1da50331bc5a None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:37.697744 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1bc5f4-e3fa-4fb3-9d74-1da50331bc5a None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:37.699267 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 386/769] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:15:37 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:15:37.699855 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1bc5f4-e3fa-4fb3-9d74-1da50331bc5a None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:37.720586 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:37.736246 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21ecc43-0d52-439e-a652-c0ce67a48b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:37.736964 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21ecc43-0d52-439e-a652-c0ce67a48b1d None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:37.737193 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21ecc43-0d52-439e-a652-c0ce67a48b1d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:37.737475 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21ecc43-0d52-439e-a652-c0ce67a48b1d None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:37.739260 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21ecc43-0d52-439e-a652-c0ce67a48b1d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:37.739260 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21ecc43-0d52-439e-a652-c0ce67a48b1d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:37.739980 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21ecc43-0d52-439e-a652-c0ce67a48b1d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:37.750172 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 384/770] 10.222.0.33 () {56 vars in 1124 bytes} [Fri Jul 15 19:15:37 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:37.750172 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3448149-18bf-4856-88e3-366f31904ddd None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:37.750172 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3448149-18bf-4856-88e3-366f31904ddd None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:37.750172 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3448149-18bf-4856-88e3-366f31904ddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:37.765506 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-d3448149-18bf-4856-88e3-366f31904ddd None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:15:37.766724 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d3448149-18bf-4856-88e3-366f31904ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:15:37.839241 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1bc5f4-e3fa-4fb3-9d74-1da50331bc5a None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:37.849789 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 385/771] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:15:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:15:37.849789 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3448149-18bf-4856-88e3-366f31904ddd None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:37.900671 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fccc5be-6b4c-4276-88c6-6bf483f09286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:37.901743 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fccc5be-6b4c-4276-88c6-6bf483f09286 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:37.902275 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fccc5be-6b4c-4276-88c6-6bf483f09286 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:37.902805 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fccc5be-6b4c-4276-88c6-6bf483f09286 None admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:37.903798 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fccc5be-6b4c-4276-88c6-6bf483f09286 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:37.904317 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fccc5be-6b4c-4276-88c6-6bf483f09286 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:37.906027 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fccc5be-6b4c-4276-88c6-6bf483f09286 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:37.924795 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 387/772] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jul 15 19:15:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:37.926029 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fccc5be-6b4c-4276-88c6-6bf483f09286 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:37.946081 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:37.955649 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3448149-18bf-4856-88e3-366f31904ddd None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:37.991345 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8189eea-3b08-4098-8580-a6f2cee34ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:37.992008 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8189eea-3b08-4098-8580-a6f2cee34ffd None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:37.992289 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8189eea-3b08-4098-8580-a6f2cee34ffd None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:37.992562 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8189eea-3b08-4098-8580-a6f2cee34ffd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:37.993348 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8189eea-3b08-4098-8580-a6f2cee34ffd None admin] RBAC: Adding query filter params (interface=public, service_id=16d91132cbf8475a8f7bef4882e8d2f1, region_id=RegionOne) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:37.993673 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8189eea-3b08-4098-8580-a6f2cee34ffd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:37.995071 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8189eea-3b08-4098-8580-a6f2cee34ffd None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:38.007118 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 386/773] 10.222.0.33 () {56 vars in 1251 bytes} [Fri Jul 15 19:15:37 2022] GET /identity/v3/endpoints?service_id=16d91132cbf8475a8f7bef4882e8d2f1&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:38.007118 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8189eea-3b08-4098-8580-a6f2cee34ffd None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:38.026162 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:38.038154 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fccc5be-6b4c-4276-88c6-6bf483f09286 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:38.074632 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705c3338-260e-4848-a398-1c096259f28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:38.075856 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705c3338-260e-4848-a398-1c096259f28c None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:38.076422 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705c3338-260e-4848-a398-1c096259f28c None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:38.076906 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705c3338-260e-4848-a398-1c096259f28c None admin] PATH_INFO: `/v3/services/network` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:38.077799 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705c3338-260e-4848-a398-1c096259f28c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:38.079382 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705c3338-260e-4848-a398-1c096259f28c None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:38.092499 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-705c3338-260e-4848-a398-1c096259f28c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 15 19:15:38.092499 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:38.092499 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 19:15:38.092499 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 19:15:38.092499 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 19:15:38.092499 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 19:15:38.092499 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 19:15:38.092499 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:15:38.092499 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 15 19:15:38.092499 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:38.092499 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 19:15:38.092499 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:38.092499 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:38.092499 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:38.092499 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:38.092499 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:38.092499 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:38.093525 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:38.093525 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:38.093525 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:38.093525 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:38.093525 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:38.093525 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:38.093525 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 19:15:38.093525 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 19:15:38.093525 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 19:15:38.093525 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 19:15:38.093525 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:38.093525 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:38.093525 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:38.093525 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:38.093525 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:38.093525 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:38.094403 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:38.094403 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:38.094403 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:38.094403 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:38.094403 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:38.094403 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:38.094403 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:38.094403 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:38.094403 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:38.094403 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:38.094403 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 19:15:38.094403 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:15:38.094403 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 15 19:15:38.094403 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:38.101878 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 388/774] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 15 19:15:37 2022] GET /identity/v3/services/network => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:38.103065 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705c3338-260e-4848-a398-1c096259f28c None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:38.109437 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8189eea-3b08-4098-8580-a6f2cee34ffd None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:38.145982 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a953cf1-241c-4c12-bf51-25bc58c133ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:38.147128 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a953cf1-241c-4c12-bf51-25bc58c133ec None admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:38.147406 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a953cf1-241c-4c12-bf51-25bc58c133ec None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:38.147773 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a953cf1-241c-4c12-bf51-25bc58c133ec None admin] PATH_INFO: `/v3/projects/55a756ccaef14cb49b26b0d68dd0b466/users/7673a3b5c0324e07a9535ac70ed7a491/roles/4318f86ba9f74cd4a0bef1c311276657` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:38.149916 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a953cf1-241c-4c12-bf51-25bc58c133ec None admin] RBAC: Authorizing `identity:create_grant(project_id=55a756ccaef14cb49b26b0d68dd0b466, user_id=7673a3b5c0324e07a9535ac70ed7a491, role_id=4318f86ba9f74cd4a0bef1c311276657)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:38.151480 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a953cf1-241c-4c12-bf51-25bc58c133ec None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:38.191166 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:38.211424 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705c3338-260e-4848-a398-1c096259f28c None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:38.237743 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 387/775] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jul 15 19:15:38 2022] PUT /identity/v3/projects/55a756ccaef14cb49b26b0d68dd0b466/users/7673a3b5c0324e07a9535ac70ed7a491/roles/4318f86ba9f74cd4a0bef1c311276657 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:15:38.239274 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a953cf1-241c-4c12-bf51-25bc58c133ec None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:38.246485 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c05e47-1b91-4786-ae10-3798ad1064c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:38.247224 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c05e47-1b91-4786-ae10-3798ad1064c8 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:38.247516 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c05e47-1b91-4786-ae10-3798ad1064c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:38.248211 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c05e47-1b91-4786-ae10-3798ad1064c8 None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:38.249421 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c05e47-1b91-4786-ae10-3798ad1064c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:38.251504 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c05e47-1b91-4786-ae10-3798ad1064c8 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:38.267467 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 389/776] 10.222.0.33 () {56 vars in 1087 bytes} [Fri Jul 15 19:15:38 2022] GET /identity/v3/services => generated 772 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:38.269408 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec0aa2-4f42-4eb3-aec6-3a74cab4b298 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:38.271098 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec0aa2-4f42-4eb3-aec6-3a74cab4b298 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:38.272653 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec0aa2-4f42-4eb3-aec6-3a74cab4b298 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:38.275219 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 390/777] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:15:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:15:38.282736 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d931e2d-13c3-4155-a32a-eb76b3972c1e None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:38.283391 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d931e2d-13c3-4155-a32a-eb76b3972c1e None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:38.283879 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d931e2d-13c3-4155-a32a-eb76b3972c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:38.318214 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:38.322726 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-4d931e2d-13c3-4155-a32a-eb76b3972c1e None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:15:38.327475 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-4d931e2d-13c3-4155-a32a-eb76b3972c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:15:38.443177 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a953cf1-241c-4c12-bf51-25bc58c133ec None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:38.479189 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 391/778] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:15:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:15:38.487460 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81f56de-7b21-4465-9708-54087e0b2085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:38.488136 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81f56de-7b21-4465-9708-54087e0b2085 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:38.488421 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81f56de-7b21-4465-9708-54087e0b2085 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:38.488699 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81f56de-7b21-4465-9708-54087e0b2085 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:38.491144 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81f56de-7b21-4465-9708-54087e0b2085 None admin] RBAC: Adding query filter params (name=network) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:38.491144 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81f56de-7b21-4465-9708-54087e0b2085 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:38.491642 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81f56de-7b21-4465-9708-54087e0b2085 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:38.498485 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 388/779] 10.222.0.33 () {56 vars in 1112 bytes} [Fri Jul 15 19:15:38 2022] GET /identity/v3/services?name=network => generated 124 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:38.506886 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81f56de-7b21-4465-9708-54087e0b2085 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:38.507054 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d931e2d-13c3-4155-a32a-eb76b3972c1e None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:38.636024 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81f56de-7b21-4465-9708-54087e0b2085 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:38.638445 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:38.676162 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc076d60-dbdb-4686-9fff-fc66f42c424c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:38.677263 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc076d60-dbdb-4686-9fff-fc66f42c424c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:38.677752 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc076d60-dbdb-4686-9fff-fc66f42c424c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:38.678253 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc076d60-dbdb-4686-9fff-fc66f42c424c None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:38.679199 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc076d60-dbdb-4686-9fff-fc66f42c424c None admin] RBAC: Adding query filter params (type=network) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:38.679705 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc076d60-dbdb-4686-9fff-fc66f42c424c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:38.681283 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc076d60-dbdb-4686-9fff-fc66f42c424c None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:38.687411 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 389/780] 10.222.0.33 () {56 vars in 1112 bytes} [Fri Jul 15 19:15:38 2022] GET /identity/v3/services?type=network => generated 350 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:38.694095 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc076d60-dbdb-4686-9fff-fc66f42c424c None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:38.826853 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:38.830895 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc076d60-dbdb-4686-9fff-fc66f42c424c None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:38.873413 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65561a14-60ac-4d3e-8ed7-2a68147cc6ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:38.874492 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65561a14-60ac-4d3e-8ed7-2a68147cc6ae None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:38.874982 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65561a14-60ac-4d3e-8ed7-2a68147cc6ae None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:38.875486 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65561a14-60ac-4d3e-8ed7-2a68147cc6ae None admin] PATH_INFO: `/v3/endpoints` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:38.876494 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65561a14-60ac-4d3e-8ed7-2a68147cc6ae None admin] RBAC: Adding query filter params (interface=public, service_id=3fc8819cd8594dd2a316d41946acacfc, region_id=RegionOne) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:38.877753 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65561a14-60ac-4d3e-8ed7-2a68147cc6ae None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:38.880687 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65561a14-60ac-4d3e-8ed7-2a68147cc6ae None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:38.887227 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 390/781] 10.222.0.33 () {56 vars in 1251 bytes} [Fri Jul 15 19:15:38 2022] GET /identity/v3/endpoints?service_id=3fc8819cd8594dd2a316d41946acacfc&interface=public®ion_id=RegionOne => generated 194 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:38.892936 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65561a14-60ac-4d3e-8ed7-2a68147cc6ae None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:38.899958 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d931e2d-13c3-4155-a32a-eb76b3972c1e None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:38.925947 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:38.968685 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9301b4fb-6ff3-4421-97fb-6b80cafd98c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:38.969404 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9301b4fb-6ff3-4421-97fb-6b80cafd98c4 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:38.969680 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9301b4fb-6ff3-4421-97fb-6b80cafd98c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:38.969980 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9301b4fb-6ff3-4421-97fb-6b80cafd98c4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:38.994527 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9301b4fb-6ff3-4421-97fb-6b80cafd98c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:15:39.002105 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-9301b4fb-6ff3-4421-97fb-6b80cafd98c4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:15:39.002105 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:39.002105 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:39.002105 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:39.002105 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:39.002105 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:39.002105 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:39.002105 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:39.002105 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:39.002105 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:39.002105 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:39.002105 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:39.002105 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:15:39.002105 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:15:39.002105 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:15:39.002105 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:15:39.003072 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:15:39.003072 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:15:39.003072 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:15:39.003072 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:15:39.003072 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:39.003072 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:39.003072 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:39.003072 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:39.003072 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:39.003072 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:39.003072 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:39.003072 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:39.003072 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:39.003072 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:39.003072 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:39.003072 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:39.003926 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:39.003926 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:39.003926 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:39.003926 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:39.003926 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:15:39.003926 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:15:39.003926 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:15:39.003926 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:15:39.003926 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:15:39.003926 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:15:39.003926 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:15:39.003926 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:39.003926 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 392/782] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jul 15 19:15:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 497 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:39.016161 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9301b4fb-6ff3-4421-97fb-6b80cafd98c4 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:39.042960 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65561a14-60ac-4d3e-8ed7-2a68147cc6ae None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:39.079953 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72804b36-a086-4c98-9d96-0e05b5a824b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:39.080624 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72804b36-a086-4c98-9d96-0e05b5a824b4 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:39.080902 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72804b36-a086-4c98-9d96-0e05b5a824b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:39.081174 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72804b36-a086-4c98-9d96-0e05b5a824b4 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:39.081957 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72804b36-a086-4c98-9d96-0e05b5a824b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:39.084096 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72804b36-a086-4c98-9d96-0e05b5a824b4 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:39.097933 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 391/783] 10.222.0.33 () {56 vars in 1087 bytes} [Fri Jul 15 19:15:38 2022] GET /identity/v3/services => generated 772 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:15:39.157138 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9301b4fb-6ff3-4421-97fb-6b80cafd98c4 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:39.198892 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab015963-9b4c-4da7-ab8f-8a2213e141a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:39.200005 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab015963-9b4c-4da7-ab8f-8a2213e141a1 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:39.200495 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab015963-9b4c-4da7-ab8f-8a2213e141a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:39.201024 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab015963-9b4c-4da7-ab8f-8a2213e141a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:39.202063 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab015963-9b4c-4da7-ab8f-8a2213e141a1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:39.202594 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab015963-9b4c-4da7-ab8f-8a2213e141a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:39.204247 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab015963-9b4c-4da7-ab8f-8a2213e141a1 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:39.206180 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:39.214842 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 393/784] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 15 19:15:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:39.220811 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72804b36-a086-4c98-9d96-0e05b5a824b4 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:39.348754 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:39.358183 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72804b36-a086-4c98-9d96-0e05b5a824b4 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:39.401959 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e81706-58a7-432e-a6df-25ddccf2e148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:39.403077 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e81706-58a7-432e-a6df-25ddccf2e148 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:39.403544 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e81706-58a7-432e-a6df-25ddccf2e148 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:39.404033 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e81706-58a7-432e-a6df-25ddccf2e148 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:39.416464 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e81706-58a7-432e-a6df-25ddccf2e148 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:39.418232 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e81706-58a7-432e-a6df-25ddccf2e148 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:39.436509 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-09e81706-58a7-432e-a6df-25ddccf2e148 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:39.436509 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:39.436509 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:39.436509 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:39.436509 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:39.436509 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:39.436509 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:39.436509 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:39.436509 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:39.436509 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:39.436509 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:39.436509 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:39.436509 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:15:39.436509 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:15:39.436509 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:15:39.437432 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:15:39.437432 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:39.437432 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:39.437432 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:39.437432 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:39.437432 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:39.437432 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:39.437432 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:39.437432 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:39.437432 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:39.437432 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:39.437432 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:39.437432 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:39.437432 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:39.437432 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:39.437432 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:39.437432 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:39.438373 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:15:39.438373 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:15:39.438373 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:39.438373 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:39.440260 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 392/785] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:15:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:39.446583 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab015963-9b4c-4da7-ab8f-8a2213e141a1 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:39.552534 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab015963-9b4c-4da7-ab8f-8a2213e141a1 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:39.599047 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46069d72-6c87-4642-a564-cc29fd5d07ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:39.599828 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46069d72-6c87-4642-a564-cc29fd5d07ab None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:39.600142 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46069d72-6c87-4642-a564-cc29fd5d07ab None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:39.600418 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46069d72-6c87-4642-a564-cc29fd5d07ab None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:39.601154 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46069d72-6c87-4642-a564-cc29fd5d07ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:39.601453 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46069d72-6c87-4642-a564-cc29fd5d07ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:39.602838 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46069d72-6c87-4642-a564-cc29fd5d07ab None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:39.616900 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 394/786] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:15:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:39.622303 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e81706-58a7-432e-a6df-25ddccf2e148 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:39.679044 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c233a50b-f139-453e-b0db-3f853db911ea None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:39.679574 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c233a50b-f139-453e-b0db-3f853db911ea None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:39.680037 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c233a50b-f139-453e-b0db-3f853db911ea None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:39.681867 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 395/787] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:15:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 19:15:39.687599 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e205eba6-09c7-4177-973f-c68b4efc87d0 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:39.688064 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e205eba6-09c7-4177-973f-c68b4efc87d0 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:39.688515 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e205eba6-09c7-4177-973f-c68b4efc87d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:39.706420 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-e205eba6-09c7-4177-973f-c68b4efc87d0 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:15:39.707940 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-e205eba6-09c7-4177-973f-c68b4efc87d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:15:39.735492 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e81706-58a7-432e-a6df-25ddccf2e148 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:39.765351 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:39.774114 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001f388e-798b-4cf6-b8ec-166ef1cc831e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:39.780308 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f388e-798b-4cf6-b8ec-166ef1cc831e None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:39.780308 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f388e-798b-4cf6-b8ec-166ef1cc831e None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:39.780308 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f388e-798b-4cf6-b8ec-166ef1cc831e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:39.783798 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001f388e-798b-4cf6-b8ec-166ef1cc831e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:39.785427 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001f388e-798b-4cf6-b8ec-166ef1cc831e None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:39.792878 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 396/788] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:15:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 19:15:39.814640 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-001f388e-798b-4cf6-b8ec-166ef1cc831e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 15 19:15:39.814640 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:39.814640 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:39.814640 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:39.814640 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:39.814640 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:39.814640 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:39.814640 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:39.814640 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:39.814640 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:39.814640 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:39.814640 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:39.814640 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 19:15:39.814640 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 19:15:39.814640 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 19:15:39.814640 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 19:15:39.815606 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:39.815606 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:39.815606 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:15:39.815606 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:15:39.815606 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:15:39.815606 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:15:39.815606 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:39.815606 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:39.815606 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:39.815606 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:39.815606 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:39.815606 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:39.815606 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:39.815606 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:39.815606 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:39.815606 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:39.815606 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:39.815606 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:39.816597 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:39.816597 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:39.816597 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 19:15:39.816597 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 19:15:39.816597 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 19:15:39.816597 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 19:15:39.816597 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 19:15:39.816597 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 19:15:39.816597 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 15 19:15:39.816597 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:39.826754 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 393/789] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jul 15 19:15:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:39.833220 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e205eba6-09c7-4177-973f-c68b4efc87d0 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:39.836359 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001f388e-798b-4cf6-b8ec-166ef1cc831e None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:39.854558 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:39.999256 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e205eba6-09c7-4177-973f-c68b4efc87d0 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:40.041199 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fcac8b-9b6b-4779-9151-6163c5c9a1c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:40.042081 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fcac8b-9b6b-4779-9151-6163c5c9a1c6 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:40.042366 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fcac8b-9b6b-4779-9151-6163c5c9a1c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:40.042638 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fcac8b-9b6b-4779-9151-6163c5c9a1c6 None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:40.043478 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fcac8b-9b6b-4779-9151-6163c5c9a1c6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:40.043784 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fcac8b-9b6b-4779-9151-6163c5c9a1c6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:40.045207 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fcac8b-9b6b-4779-9151-6163c5c9a1c6 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:40.047740 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001f388e-798b-4cf6-b8ec-166ef1cc831e None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:40.063648 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 397/790] 10.222.0.33 () {56 vars in 1135 bytes} [Fri Jul 15 19:15:39 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:40.074959 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fcac8b-9b6b-4779-9151-6163c5c9a1c6 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:40.102859 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d61f25c-34ac-4d6b-8abc-448e30e96ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:40.103939 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d61f25c-34ac-4d6b-8abc-448e30e96ea2 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:40.104224 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d61f25c-34ac-4d6b-8abc-448e30e96ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:40.104929 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d61f25c-34ac-4d6b-8abc-448e30e96ea2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:40.130534 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:40.131844 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d61f25c-34ac-4d6b-8abc-448e30e96ea2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:15:40.144899 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-7d61f25c-34ac-4d6b-8abc-448e30e96ea2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:15:40.144899 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:40.144899 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:40.144899 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:40.144899 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:40.144899 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:40.144899 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:40.144899 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:40.144899 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:40.144899 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:40.144899 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:40.144899 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:40.144899 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:15:40.144899 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:15:40.144899 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:15:40.144899 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:15:40.149954 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:15:40.149954 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:15:40.149954 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:15:40.149954 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:15:40.149954 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:40.149954 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:40.149954 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:40.149954 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:40.149954 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:40.149954 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:40.149954 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:40.149954 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:40.149954 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:40.149954 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:40.149954 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:40.149954 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:40.150818 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:40.150818 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:40.150818 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:40.150818 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:40.150818 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:15:40.150818 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:15:40.150818 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:15:40.150818 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:15:40.150818 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:15:40.150818 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:15:40.150818 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:15:40.150818 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:40.150818 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 394/791] 10.222.0.33 () {56 vars in 1102 bytes} [Fri Jul 15 19:15:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:15:40.153217 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d61f25c-34ac-4d6b-8abc-448e30e96ea2 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:40.184881 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fcac8b-9b6b-4779-9151-6163c5c9a1c6 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:40.222375 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcf548e-049e-457d-a01f-89852928698b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:40.223539 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcf548e-049e-457d-a01f-89852928698b None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:40.223827 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcf548e-049e-457d-a01f-89852928698b None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:40.224101 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcf548e-049e-457d-a01f-89852928698b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:40.235825 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcf548e-049e-457d-a01f-89852928698b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:40.237357 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcf548e-049e-457d-a01f-89852928698b None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:40.256822 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-dbcf548e-049e-457d-a01f-89852928698b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:40.256822 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:40.256822 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:40.256822 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:40.256822 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:40.256822 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:40.256822 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:40.256822 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:40.256822 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:40.256822 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:40.256822 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:40.256822 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:40.256822 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:15:40.256822 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:15:40.256822 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:15:40.257822 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:15:40.257822 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:40.257822 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:40.257822 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:40.257822 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:40.257822 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:40.257822 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:40.257822 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:40.257822 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:40.257822 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:40.257822 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:40.257822 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:40.257822 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:40.257822 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:40.257822 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:40.257822 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:40.257822 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:40.258734 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:15:40.258734 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:15:40.258734 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:40.258734 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:40.258734 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 398/792] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:15:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:40.264445 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcf548e-049e-457d-a01f-89852928698b None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:40.313980 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:40.385387 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d61f25c-34ac-4d6b-8abc-448e30e96ea2 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:40.437808 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:40.443953 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcf548e-049e-457d-a01f-89852928698b None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:40.456880 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ad1a00-659d-4135-8921-188e97ce5897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:40.458126 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ad1a00-659d-4135-8921-188e97ce5897 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:40.458646 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ad1a00-659d-4135-8921-188e97ce5897 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:40.459104 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ad1a00-659d-4135-8921-188e97ce5897 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:40.460035 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ad1a00-659d-4135-8921-188e97ce5897 None admin] RBAC: Adding query filter params (name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:40.460512 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ad1a00-659d-4135-8921-188e97ce5897 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:40.462102 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ad1a00-659d-4135-8921-188e97ce5897 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:40.479184 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efcffc2-6f21-4201-a487-d22394cbe1d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:40.479850 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efcffc2-6f21-4201-a487-d22394cbe1d2 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:40.480131 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efcffc2-6f21-4201-a487-d22394cbe1d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:40.480406 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efcffc2-6f21-4201-a487-d22394cbe1d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:40.481133 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efcffc2-6f21-4201-a487-d22394cbe1d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:40.481304 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 395/793] 10.222.0.33 () {56 vars in 1103 bytes} [Fri Jul 15 19:15:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:40.481562 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efcffc2-6f21-4201-a487-d22394cbe1d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:40.483146 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efcffc2-6f21-4201-a487-d22394cbe1d2 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:40.486493 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ad1a00-659d-4135-8921-188e97ce5897 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:40.495851 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 399/794] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:15:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:15:40.500889 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efcffc2-6f21-4201-a487-d22394cbe1d2 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:40.632858 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efcffc2-6f21-4201-a487-d22394cbe1d2 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:40.668710 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55789bcd-d064-4c1f-a0ce-1aa071de8b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:40.669555 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55789bcd-d064-4c1f-a0ce-1aa071de8b1c None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:40.669882 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55789bcd-d064-4c1f-a0ce-1aa071de8b1c None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:40.670162 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55789bcd-d064-4c1f-a0ce-1aa071de8b1c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:40.677816 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55789bcd-d064-4c1f-a0ce-1aa071de8b1c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:40.679254 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55789bcd-d064-4c1f-a0ce-1aa071de8b1c None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:40.693907 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-55789bcd-d064-4c1f-a0ce-1aa071de8b1c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:15:40.693907 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:40.693907 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:40.693907 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:40.693907 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:40.693907 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:40.693907 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:40.693907 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:40.693907 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:40.693907 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:40.693907 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:40.693907 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:40.693907 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 19:15:40.693907 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 19:15:40.693907 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 19:15:40.694868 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 19:15:40.694868 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:40.694868 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:40.694868 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:40.694868 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:40.694868 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:40.694868 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:40.694868 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:40.694868 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:40.694868 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:40.694868 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:40.694868 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:40.694868 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:40.694868 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:40.694868 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:40.694868 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:40.694868 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:40.695755 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 19:15:40.695755 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 19:15:40.695755 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:15:40.695755 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:15:40.695755 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:15:40.695755 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:15:40.695755 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:15:40.695755 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:40.695755 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 400/795] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 15 19:15:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:15:40.700517 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ad1a00-659d-4135-8921-188e97ce5897 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:40.702720 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55789bcd-d064-4c1f-a0ce-1aa071de8b1c None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:40.734564 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:40.754960 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:40.763487 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94785fd9-b620-4d4e-b2a9-676c5a8fd15e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:40.764291 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94785fd9-b620-4d4e-b2a9-676c5a8fd15e None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:40.764574 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94785fd9-b620-4d4e-b2a9-676c5a8fd15e None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:40.764849 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94785fd9-b620-4d4e-b2a9-676c5a8fd15e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:40.777688 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94785fd9-b620-4d4e-b2a9-676c5a8fd15e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:40.779385 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94785fd9-b620-4d4e-b2a9-676c5a8fd15e None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:40.798431 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-94785fd9-b620-4d4e-b2a9-676c5a8fd15e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:40.798431 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:40.798431 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:40.798431 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:40.798431 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:40.798431 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:40.798431 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:40.798431 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:40.798431 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:40.798431 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:40.798431 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:40.798431 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:40.798431 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:15:40.798431 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:15:40.798431 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:15:40.799425 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:15:40.799425 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:40.799425 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:40.799425 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:40.799425 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:40.799425 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:40.799425 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:40.799425 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:40.799425 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:40.799425 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:40.799425 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:40.799425 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:40.799425 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:40.799425 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:40.799425 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:40.799425 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:40.799425 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:40.800320 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:15:40.800320 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:15:40.800320 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:40.800320 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:40.800320 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 396/796] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:15:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:15:40.801482 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ca9eee-4ddc-4504-a816-5c225e4a3d18 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:40.801611 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ca9eee-4ddc-4504-a816-5c225e4a3d18 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:40.802495 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ca9eee-4ddc-4504-a816-5c225e4a3d18 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:40.809964 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 397/797] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:15:40 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:15:40.811432 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afbbc9a-8e94-4d32-9233-b8089d22fe10 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:40.811719 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afbbc9a-8e94-4d32-9233-b8089d22fe10 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:40.812330 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afbbc9a-8e94-4d32-9233-b8089d22fe10 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:40.813343 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 398/798] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:15:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:15:40.814265 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afbbc9a-8e94-4d32-9233-b8089d22fe10 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:40.841903 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55789bcd-d064-4c1f-a0ce-1aa071de8b1c None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:40.880187 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4625934f-b00c-43ee-8649-f2f12907c0f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:40.880859 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4625934f-b00c-43ee-8649-f2f12907c0f7 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:40.881139 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4625934f-b00c-43ee-8649-f2f12907c0f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:40.881592 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4625934f-b00c-43ee-8649-f2f12907c0f7 None admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:40.882279 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4625934f-b00c-43ee-8649-f2f12907c0f7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:40.882530 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4625934f-b00c-43ee-8649-f2f12907c0f7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:40.884009 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4625934f-b00c-43ee-8649-f2f12907c0f7 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:40.896504 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 401/799] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jul 15 19:15:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:40.897573 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e46fc18-9d35-4268-8da8-201a24db88c6 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:40.897852 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e46fc18-9d35-4268-8da8-201a24db88c6 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:40.898134 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e46fc18-9d35-4268-8da8-201a24db88c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:40.915424 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-3e46fc18-9d35-4268-8da8-201a24db88c6 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:15:40.916760 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-3e46fc18-9d35-4268-8da8-201a24db88c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:15:40.931447 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afbbc9a-8e94-4d32-9233-b8089d22fe10 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:40.964997 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eac67c1-8356-4def-8f49-223137bbf6f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:40.965661 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eac67c1-8356-4def-8f49-223137bbf6f5 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:40.965975 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eac67c1-8356-4def-8f49-223137bbf6f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:40.966249 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eac67c1-8356-4def-8f49-223137bbf6f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:40.966970 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eac67c1-8356-4def-8f49-223137bbf6f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:40.967276 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eac67c1-8356-4def-8f49-223137bbf6f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:40.968642 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eac67c1-8356-4def-8f49-223137bbf6f5 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:40.980298 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 399/800] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:15:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:40.981500 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fab3195-27eb-44d5-9691-d71e0d013acb None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:40.981804 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fab3195-27eb-44d5-9691-d71e0d013acb None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:40.982090 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fab3195-27eb-44d5-9691-d71e0d013acb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:40.995467 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 402/801] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:15:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:15:40.996563 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e46fc18-9d35-4268-8da8-201a24db88c6 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:41.006770 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-4fab3195-27eb-44d5-9691-d71e0d013acb None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:15:41.007970 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-4fab3195-27eb-44d5-9691-d71e0d013acb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:15:41.012571 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:41.043221 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:41.089786 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 400/802] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:15:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:15:41.089786 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fab3195-27eb-44d5-9691-d71e0d013acb None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:41.106572 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e46fc18-9d35-4268-8da8-201a24db88c6 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:41.141832 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c947b666-41f4-4dcf-ae2d-455b6beab507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:41.142646 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c947b666-41f4-4dcf-ae2d-455b6beab507 None admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:41.142926 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c947b666-41f4-4dcf-ae2d-455b6beab507 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:41.143200 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c947b666-41f4-4dcf-ae2d-455b6beab507 None admin] PATH_INFO: `/v3/projects/55a756ccaef14cb49b26b0d68dd0b466/users/ef76f7fe39a3499aaeebf85c176804f3/roles/4318f86ba9f74cd4a0bef1c311276657` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:41.145326 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c947b666-41f4-4dcf-ae2d-455b6beab507 None admin] RBAC: Authorizing `identity:create_grant(project_id=55a756ccaef14cb49b26b0d68dd0b466, user_id=ef76f7fe39a3499aaeebf85c176804f3, role_id=4318f86ba9f74cd4a0bef1c311276657)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:41.147048 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c947b666-41f4-4dcf-ae2d-455b6beab507 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:41.195785 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fab3195-27eb-44d5-9691-d71e0d013acb None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:41.214597 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 403/803] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jul 15 19:15:40 2022] PUT /identity/v3/projects/55a756ccaef14cb49b26b0d68dd0b466/users/ef76f7fe39a3499aaeebf85c176804f3/roles/4318f86ba9f74cd4a0bef1c311276657 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:15:41.215699 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c947b666-41f4-4dcf-ae2d-455b6beab507 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:41.240494 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f70d7d-c62f-44d6-a0db-c608d5a52293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:41.241657 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f70d7d-c62f-44d6-a0db-c608d5a52293 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:41.242069 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f70d7d-c62f-44d6-a0db-c608d5a52293 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:41.242520 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f70d7d-c62f-44d6-a0db-c608d5a52293 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:41.251781 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f70d7d-c62f-44d6-a0db-c608d5a52293 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:41.253722 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f70d7d-c62f-44d6-a0db-c608d5a52293 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:41.272049 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-c3f70d7d-c62f-44d6-a0db-c608d5a52293 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 15 19:15:41.272049 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:41.272049 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:41.272049 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:41.272049 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:41.272049 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:41.272049 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:41.272049 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:41.272049 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:41.272049 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:41.272049 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:41.272049 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:41.272049 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 19:15:41.272049 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 19:15:41.272049 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 19:15:41.272049 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 19:15:41.273082 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:41.273082 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:41.273082 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:15:41.273082 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:15:41.273082 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:15:41.273082 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:15:41.273082 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:41.273082 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:41.273082 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:41.273082 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:41.273082 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:41.273082 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:41.273082 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:41.273082 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:41.273082 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:41.273082 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:41.273082 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:41.273082 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:41.274097 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:41.274097 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:41.274097 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 19:15:41.274097 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 19:15:41.274097 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 19:15:41.274097 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 19:15:41.274097 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 19:15:41.274097 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 19:15:41.274097 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 15 19:15:41.274097 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:41.278202 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 401/804] 10.222.0.33 () {56 vars in 1117 bytes} [Fri Jul 15 19:15:41 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:41.279369 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f70d7d-c62f-44d6-a0db-c608d5a52293 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:41.310725 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:41.317110 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:41.366384 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c947b666-41f4-4dcf-ae2d-455b6beab507 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:41.407486 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2220111a-28a3-4423-8d0d-ad04df2d018a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:41.408224 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2220111a-28a3-4423-8d0d-ad04df2d018a None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:41.408506 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2220111a-28a3-4423-8d0d-ad04df2d018a None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:41.408780 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2220111a-28a3-4423-8d0d-ad04df2d018a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:41.409438 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2220111a-28a3-4423-8d0d-ad04df2d018a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:41.411048 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2220111a-28a3-4423-8d0d-ad04df2d018a None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:41.416175 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f70d7d-c62f-44d6-a0db-c608d5a52293 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:41.423978 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-2220111a-28a3-4423-8d0d-ad04df2d018a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 15 19:15:41.423978 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:41.423978 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 19:15:41.423978 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 19:15:41.423978 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 19:15:41.423978 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 19:15:41.423978 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 19:15:41.423978 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:15:41.423978 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 15 19:15:41.423978 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:41.423978 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 19:15:41.423978 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:41.423978 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:41.423978 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:41.423978 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:41.423978 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:41.424935 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:41.424935 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:41.424935 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:41.424935 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:41.424935 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:41.424935 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:41.424935 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:41.424935 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 19:15:41.424935 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 19:15:41.424935 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 19:15:41.424935 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 19:15:41.424935 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:41.424935 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:41.424935 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:41.424935 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:41.424935 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:41.425832 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:41.425832 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:41.425832 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:41.425832 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:41.425832 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:41.425832 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:41.425832 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:41.425832 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:41.425832 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:41.425832 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:41.425832 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:41.425832 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 19:15:41.425832 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:15:41.425832 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 15 19:15:41.425832 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:41.425832 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 404/805] 10.222.0.33 () {56 vars in 1129 bytes} [Fri Jul 15 19:15:41 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:41.427247 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2220111a-28a3-4423-8d0d-ad04df2d018a None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:41.467649 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b80a323-58f8-4dff-8f52-3d0c773b5102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:41.468911 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b80a323-58f8-4dff-8f52-3d0c773b5102 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:41.469321 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b80a323-58f8-4dff-8f52-3d0c773b5102 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:41.469602 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b80a323-58f8-4dff-8f52-3d0c773b5102 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:41.486711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b80a323-58f8-4dff-8f52-3d0c773b5102 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:15:41.496943 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-5b80a323-58f8-4dff-8f52-3d0c773b5102 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:15:41.496943 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:41.496943 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:41.496943 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:41.496943 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:41.496943 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:41.496943 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:41.496943 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:41.496943 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:41.496943 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:41.496943 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:41.496943 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:41.496943 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:15:41.496943 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:15:41.496943 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:15:41.496943 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:15:41.497997 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:15:41.497997 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:15:41.497997 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:15:41.497997 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:15:41.497997 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:41.497997 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:41.497997 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:41.497997 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:41.497997 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:41.497997 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:41.497997 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:41.497997 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:41.497997 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:41.497997 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:41.497997 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:41.497997 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:41.498862 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:41.498862 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:41.498862 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:41.498862 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:41.498862 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:15:41.498862 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:15:41.498862 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:15:41.498862 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:15:41.498862 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:15:41.498862 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:15:41.498862 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:15:41.498862 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:41.501857 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 402/806] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jul 15 19:15:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:41.502791 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b80a323-58f8-4dff-8f52-3d0c773b5102 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:41.550369 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2220111a-28a3-4423-8d0d-ad04df2d018a None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:41.580018 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:41.587020 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e71cdf-5215-4e16-b8b2-ccc17e85377f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:41.587903 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e71cdf-5215-4e16-b8b2-ccc17e85377f None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:41.588290 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e71cdf-5215-4e16-b8b2-ccc17e85377f None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:41.588671 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e71cdf-5215-4e16-b8b2-ccc17e85377f None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:41.589631 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e71cdf-5215-4e16-b8b2-ccc17e85377f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:41.590066 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e71cdf-5215-4e16-b8b2-ccc17e85377f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:41.591566 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e71cdf-5215-4e16-b8b2-ccc17e85377f None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:41.606170 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:41.610494 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 405/807] 10.222.0.33 () {56 vars in 1149 bytes} [Fri Jul 15 19:15:41 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:41.611409 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e71cdf-5215-4e16-b8b2-ccc17e85377f None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:41.622461 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b80a323-58f8-4dff-8f52-3d0c773b5102 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:41.659849 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413eb133-ed5d-4c76-adec-5f4b86622ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:41.660577 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413eb133-ed5d-4c76-adec-5f4b86622ef1 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:41.660929 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413eb133-ed5d-4c76-adec-5f4b86622ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:41.661278 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413eb133-ed5d-4c76-adec-5f4b86622ef1 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:41.662114 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413eb133-ed5d-4c76-adec-5f4b86622ef1 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:41.662492 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413eb133-ed5d-4c76-adec-5f4b86622ef1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:41.663962 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413eb133-ed5d-4c76-adec-5f4b86622ef1 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:41.673453 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 403/808] 10.222.0.33 () {56 vars in 1124 bytes} [Fri Jul 15 19:15:41 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:41.674479 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413eb133-ed5d-4c76-adec-5f4b86622ef1 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:41.746927 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e71cdf-5215-4e16-b8b2-ccc17e85377f None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:41.788239 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413eb133-ed5d-4c76-adec-5f4b86622ef1 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:41.788920 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8077072-2828-482f-ae1a-7a670b446f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:41.789673 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8077072-2828-482f-ae1a-7a670b446f86 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:41.790052 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8077072-2828-482f-ae1a-7a670b446f86 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:41.790403 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8077072-2828-482f-ae1a-7a670b446f86 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:41.791232 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8077072-2828-482f-ae1a-7a670b446f86 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:41.791635 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8077072-2828-482f-ae1a-7a670b446f86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:41.793067 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8077072-2828-482f-ae1a-7a670b446f86 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:41.803283 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 406/809] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 15 19:15:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:41.806068 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8077072-2828-482f-ae1a-7a670b446f86 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:41.829885 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed41720-b333-4562-b9d6-de635485e9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:41.830766 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed41720-b333-4562-b9d6-de635485e9a3 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:41.831058 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed41720-b333-4562-b9d6-de635485e9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:41.831341 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed41720-b333-4562-b9d6-de635485e9a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:41.843467 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed41720-b333-4562-b9d6-de635485e9a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:41.844974 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed41720-b333-4562-b9d6-de635485e9a3 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:41.858998 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:41.862558 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-aed41720-b333-4562-b9d6-de635485e9a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:41.862558 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:41.862558 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:41.862558 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:41.862558 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:41.862558 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:41.862558 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:41.862558 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:41.862558 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:41.862558 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:41.862558 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:41.862558 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:41.862558 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:15:41.862558 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:15:41.862558 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:15:41.863449 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:15:41.863449 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:41.863449 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:41.863449 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:41.863449 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:41.863449 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:41.863449 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:41.863449 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:41.863449 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:41.863449 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:41.863449 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:41.863449 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:41.863449 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:41.863449 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:41.863449 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:41.863449 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:41.863449 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:41.864998 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:15:41.864998 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:15:41.864998 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:41.864998 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:41.864998 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 404/810] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:15:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:41.865804 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e288d821-2bb7-4129-9f49-88eae96a221a None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:41.865804 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e288d821-2bb7-4129-9f49-88eae96a221a None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:41.866071 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e288d821-2bb7-4129-9f49-88eae96a221a None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:41.869894 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 405/811] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:15:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:15:41.869894 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e288d821-2bb7-4129-9f49-88eae96a221a None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:41.888704 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:41.924325 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8077072-2828-482f-ae1a-7a670b446f86 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:41.965336 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11706efc-d13e-4501-997c-7d9aa407a9b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:41.966027 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11706efc-d13e-4501-997c-7d9aa407a9b8 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:41.966310 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11706efc-d13e-4501-997c-7d9aa407a9b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:41.966584 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11706efc-d13e-4501-997c-7d9aa407a9b8 None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:41.967293 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11706efc-d13e-4501-997c-7d9aa407a9b8 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:41.967595 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11706efc-d13e-4501-997c-7d9aa407a9b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:41.968957 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11706efc-d13e-4501-997c-7d9aa407a9b8 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:41.974402 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 407/812] 10.222.0.33 () {56 vars in 1124 bytes} [Fri Jul 15 19:15:41 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:41.975214 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11706efc-d13e-4501-997c-7d9aa407a9b8 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:41.989523 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e288d821-2bb7-4129-9f49-88eae96a221a None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:42.030965 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231b369a-268a-4f7b-8b80-a0dfb3fec570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:42.032046 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b369a-268a-4f7b-8b80-a0dfb3fec570 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:42.032480 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b369a-268a-4f7b-8b80-a0dfb3fec570 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:42.032911 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b369a-268a-4f7b-8b80-a0dfb3fec570 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:42.045461 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231b369a-268a-4f7b-8b80-a0dfb3fec570 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:42.047770 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231b369a-268a-4f7b-8b80-a0dfb3fec570 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:42.066095 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-231b369a-268a-4f7b-8b80-a0dfb3fec570 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:42.066095 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:42.066095 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:42.066095 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:42.066095 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:42.066095 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:42.066095 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:42.066095 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:42.066095 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:42.066095 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:42.066095 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:42.066095 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:42.066095 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:15:42.066095 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:15:42.066095 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:15:42.067522 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:15:42.067522 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:42.067522 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:42.067522 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:42.067522 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:42.067522 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:42.067522 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:42.067522 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:42.067522 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:42.067522 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:42.067522 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:42.067522 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:42.067522 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:42.067522 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:42.067522 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:42.067522 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:42.067522 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:42.068426 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:15:42.068426 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:15:42.068426 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:42.068426 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:42.068426 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 406/813] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:15:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:42.069277 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8031c14-234b-438a-8159-cc4401695e0f None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:42.069791 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8031c14-234b-438a-8159-cc4401695e0f None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:42.070140 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8031c14-234b-438a-8159-cc4401695e0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:42.086838 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-e8031c14-234b-438a-8159-cc4401695e0f None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:15:42.087971 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e8031c14-234b-438a-8159-cc4401695e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:15:42.091436 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11706efc-d13e-4501-997c-7d9aa407a9b8 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:42.105976 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:42.126009 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0625b26f-3d84-4aff-bd39-31ca774b0fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:42.126943 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0625b26f-3d84-4aff-bd39-31ca774b0fc2 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:42.127222 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0625b26f-3d84-4aff-bd39-31ca774b0fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:42.127495 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0625b26f-3d84-4aff-bd39-31ca774b0fc2 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:42.128601 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0625b26f-3d84-4aff-bd39-31ca774b0fc2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:42.128601 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0625b26f-3d84-4aff-bd39-31ca774b0fc2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:42.130209 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0625b26f-3d84-4aff-bd39-31ca774b0fc2 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:42.142201 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 408/814] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:15:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:42.147524 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0625b26f-3d84-4aff-bd39-31ca774b0fc2 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:42.153611 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:42.167525 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 407/815] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:15:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:15:42.168258 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8031c14-234b-438a-8159-cc4401695e0f None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:42.265229 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0625b26f-3d84-4aff-bd39-31ca774b0fc2 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:42.280100 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8031c14-234b-438a-8159-cc4401695e0f None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:42.302984 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf64c3e-20e4-4459-a3eb-33dca6b64499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:42.303647 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf64c3e-20e4-4459-a3eb-33dca6b64499 None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:42.303935 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf64c3e-20e4-4459-a3eb-33dca6b64499 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:42.304209 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf64c3e-20e4-4459-a3eb-33dca6b64499 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:42.304953 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf64c3e-20e4-4459-a3eb-33dca6b64499 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:42.306359 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf64c3e-20e4-4459-a3eb-33dca6b64499 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:42.316288 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eef2d4-6273-4363-8782-d539dae3be1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:42.317589 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eef2d4-6273-4363-8782-d539dae3be1f None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:42.317921 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eef2d4-6273-4363-8782-d539dae3be1f None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:42.318198 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eef2d4-6273-4363-8782-d539dae3be1f None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:42.318922 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68eef2d4-6273-4363-8782-d539dae3be1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:42.319226 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68eef2d4-6273-4363-8782-d539dae3be1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:42.320610 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68eef2d4-6273-4363-8782-d539dae3be1f None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:42.336843 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 408/816] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:15:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:42.336843 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eef2d4-6273-4363-8782-d539dae3be1f None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:42.382695 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:42.433880 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eef2d4-6273-4363-8782-d539dae3be1f None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:42.453814 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 409/817] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 15 19:15:42 2022] POST /identity/v3/endpoints => generated 346 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:15:42.453814 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf64c3e-20e4-4459-a3eb-33dca6b64499 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:42.491017 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d480ac-13f9-4088-8e6b-26398457d178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:42.492106 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d480ac-13f9-4088-8e6b-26398457d178 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:42.492526 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d480ac-13f9-4088-8e6b-26398457d178 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:42.492895 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d480ac-13f9-4088-8e6b-26398457d178 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:42.500833 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d480ac-13f9-4088-8e6b-26398457d178 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:42.502533 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d480ac-13f9-4088-8e6b-26398457d178 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:42.505225 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:42.520747 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-c2d480ac-13f9-4088-8e6b-26398457d178 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:15:42.520747 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:42.520747 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:42.520747 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:42.520747 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:42.520747 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:42.520747 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:42.520747 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:42.520747 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:42.520747 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:42.520747 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:42.520747 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:42.520747 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 19:15:42.520747 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 19:15:42.520747 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 19:15:42.522010 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 19:15:42.522010 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:42.522010 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:42.522010 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:42.522010 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:42.522010 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:42.522010 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:42.522010 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:42.522010 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:42.522010 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:42.522010 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:42.522010 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:42.522010 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:42.522010 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:42.522010 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:42.522010 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:42.522010 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:42.522918 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 19:15:42.522918 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 19:15:42.522918 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:15:42.522918 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:15:42.522918 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:15:42.522918 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:15:42.522918 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:15:42.522918 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:42.522918 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 409/818] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 15 19:15:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:42.524052 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d480ac-13f9-4088-8e6b-26398457d178 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:42.612340 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf64c3e-20e4-4459-a3eb-33dca6b64499 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:42.643490 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d480ac-13f9-4088-8e6b-26398457d178 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:42.647323 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:42.656580 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d65c535-3de5-4f99-b7b5-a32735c4acf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:42.657390 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d65c535-3de5-4f99-b7b5-a32735c4acf7 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:42.657781 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d65c535-3de5-4f99-b7b5-a32735c4acf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:42.657987 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d65c535-3de5-4f99-b7b5-a32735c4acf7 None admin] PATH_INFO: `/v3/services/network` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:42.658645 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d65c535-3de5-4f99-b7b5-a32735c4acf7 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:42.660013 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d65c535-3de5-4f99-b7b5-a32735c4acf7 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:42.673232 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-0d65c535-3de5-4f99-b7b5-a32735c4acf7 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 15 19:15:42.673232 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:42.673232 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 19:15:42.673232 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 19:15:42.673232 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 19:15:42.673232 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 19:15:42.673232 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 19:15:42.673232 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:15:42.673232 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 15 19:15:42.673232 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:42.673232 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 19:15:42.673232 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:42.673232 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:42.673232 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:42.673232 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:42.673232 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:42.673232 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:42.674413 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:42.674413 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:42.674413 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:42.674413 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:42.674413 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:42.674413 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:42.674413 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 19:15:42.674413 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 19:15:42.674413 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 19:15:42.674413 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 19:15:42.674413 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:42.674413 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:42.674413 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:42.674413 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:42.674413 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:42.674413 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:42.680600 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:42.680600 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:42.680600 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:42.680600 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:42.680600 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:42.680600 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:42.680600 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:42.680600 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:42.680600 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:42.680600 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:42.680600 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 19:15:42.680600 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:15:42.680600 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 15 19:15:42.680600 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:42.680600 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 410/819] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 15 19:15:42 2022] GET /identity/v3/services/network => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:42.681814 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d65c535-3de5-4f99-b7b5-a32735c4acf7 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:42.695380 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3269d923-bd06-4da3-9506-97cdaf34dd79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:42.696306 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3269d923-bd06-4da3-9506-97cdaf34dd79 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:42.696665 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3269d923-bd06-4da3-9506-97cdaf34dd79 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:42.697013 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3269d923-bd06-4da3-9506-97cdaf34dd79 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:42.704869 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3269d923-bd06-4da3-9506-97cdaf34dd79 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:42.706382 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3269d923-bd06-4da3-9506-97cdaf34dd79 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:42.723523 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-3269d923-bd06-4da3-9506-97cdaf34dd79 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 15 19:15:42.723523 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:42.723523 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:42.723523 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:42.723523 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:42.723523 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:42.723523 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:42.723523 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:42.723523 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:42.723523 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:42.723523 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:42.723523 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:42.723523 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 19:15:42.723523 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 19:15:42.723523 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 19:15:42.723523 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 19:15:42.724557 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:42.724557 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:42.724557 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:15:42.724557 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:15:42.724557 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:15:42.724557 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:15:42.724557 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:42.724557 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:42.724557 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:42.724557 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:42.724557 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:42.724557 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:42.724557 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:42.724557 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:42.724557 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:42.724557 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:42.724557 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:42.724557 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:42.725846 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:42.725846 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:42.725846 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 19:15:42.725846 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 19:15:42.725846 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 19:15:42.725846 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 19:15:42.725846 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 19:15:42.725846 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 19:15:42.725846 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 15 19:15:42.725846 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:42.725846 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 410/820] 10.222.0.33 () {56 vars in 1093 bytes} [Fri Jul 15 19:15:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:42.725846 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3269d923-bd06-4da3-9506-97cdaf34dd79 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:42.811738 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:42.820304 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d65c535-3de5-4f99-b7b5-a32735c4acf7 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:42.858467 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1896119a-b78f-4bf9-b2c3-3cc267a2488a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:42.859252 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896119a-b78f-4bf9-b2c3-3cc267a2488a None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:42.859784 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896119a-b78f-4bf9-b2c3-3cc267a2488a None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:42.859872 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896119a-b78f-4bf9-b2c3-3cc267a2488a None admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:42.860687 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1896119a-b78f-4bf9-b2c3-3cc267a2488a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:42.861002 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1896119a-b78f-4bf9-b2c3-3cc267a2488a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:42.862454 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1896119a-b78f-4bf9-b2c3-3cc267a2488a None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:42.871179 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3269d923-bd06-4da3-9506-97cdaf34dd79 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:42.880337 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 411/821] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jul 15 19:15:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:42.880337 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1896119a-b78f-4bf9-b2c3-3cc267a2488a None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:42.924755 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb7a649-f6f1-4863-b1a1-49806b1528e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:42.926214 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb7a649-f6f1-4863-b1a1-49806b1528e8 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:42.926659 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb7a649-f6f1-4863-b1a1-49806b1528e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:42.927080 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb7a649-f6f1-4863-b1a1-49806b1528e8 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:42.928387 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb7a649-f6f1-4863-b1a1-49806b1528e8 None admin] RBAC: Adding query filter params (name=network) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:42.929124 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb7a649-f6f1-4863-b1a1-49806b1528e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:42.930964 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb7a649-f6f1-4863-b1a1-49806b1528e8 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:42.937641 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 411/822] 10.222.0.33 () {56 vars in 1112 bytes} [Fri Jul 15 19:15:42 2022] GET /identity/v3/services?name=network => generated 124 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:42.939240 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb7a649-f6f1-4863-b1a1-49806b1528e8 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:42.978532 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:43.001086 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1896119a-b78f-4bf9-b2c3-3cc267a2488a None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:43.040122 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03abd685-878d-4927-8505-92591232f1c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:43.040943 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03abd685-878d-4927-8505-92591232f1c0 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:43.041330 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03abd685-878d-4927-8505-92591232f1c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:43.042141 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03abd685-878d-4927-8505-92591232f1c0 None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:43.042989 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03abd685-878d-4927-8505-92591232f1c0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:43.043403 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03abd685-878d-4927-8505-92591232f1c0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:43.045251 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03abd685-878d-4927-8505-92591232f1c0 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:43.067812 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 412/823] 10.222.0.33 () {56 vars in 1133 bytes} [Fri Jul 15 19:15:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:43.068763 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03abd685-878d-4927-8505-92591232f1c0 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:43.074414 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb7a649-f6f1-4863-b1a1-49806b1528e8 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:43.087639 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:43.114188 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69e6b1d-6038-41a3-8f12-df2b7e4e42d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:43.115107 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69e6b1d-6038-41a3-8f12-df2b7e4e42d3 None admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:43.116061 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69e6b1d-6038-41a3-8f12-df2b7e4e42d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:43.116061 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69e6b1d-6038-41a3-8f12-df2b7e4e42d3 None admin] PATH_INFO: `/v3/projects/55a756ccaef14cb49b26b0d68dd0b466/users/3d893c14620b4a129842062f14a68a78/roles/70de044d6962417593b6d03571e7502e` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:43.132385 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69e6b1d-6038-41a3-8f12-df2b7e4e42d3 None admin] RBAC: Authorizing `identity:create_grant(project_id=55a756ccaef14cb49b26b0d68dd0b466, user_id=3d893c14620b4a129842062f14a68a78, role_id=70de044d6962417593b6d03571e7502e)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:43.133979 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69e6b1d-6038-41a3-8f12-df2b7e4e42d3 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:43.187302 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03abd685-878d-4927-8505-92591232f1c0 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:43.222257 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 412/824] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jul 15 19:15:42 2022] PUT /identity/v3/projects/55a756ccaef14cb49b26b0d68dd0b466/users/3d893c14620b4a129842062f14a68a78/roles/70de044d6962417593b6d03571e7502e => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:15:43.223086 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69e6b1d-6038-41a3-8f12-df2b7e4e42d3 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:43.240391 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4de9d9f-019e-4677-a7e4-1c5f802251f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:43.241080 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4de9d9f-019e-4677-a7e4-1c5f802251f4 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:43.241442 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4de9d9f-019e-4677-a7e4-1c5f802251f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:43.241633 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4de9d9f-019e-4677-a7e4-1c5f802251f4 None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:43.242381 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4de9d9f-019e-4677-a7e4-1c5f802251f4 None admin] RBAC: Adding query filter params (type=network) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:43.242696 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4de9d9f-019e-4677-a7e4-1c5f802251f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:43.244412 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4de9d9f-019e-4677-a7e4-1c5f802251f4 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:43.258450 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 413/825] 10.222.0.33 () {56 vars in 1112 bytes} [Fri Jul 15 19:15:43 2022] GET /identity/v3/services?type=network => generated 350 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:43.262658 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4de9d9f-019e-4677-a7e4-1c5f802251f4 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:43.385473 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:43.392725 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:43.415366 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69e6b1d-6038-41a3-8f12-df2b7e4e42d3 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:43.418916 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4de9d9f-019e-4677-a7e4-1c5f802251f4 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:43.457792 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a93241-b453-493e-85b8-8dbcc8805247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:43.458028 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865ff85e-f472-4b3f-9f58-330dbef6d0e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:43.458446 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865ff85e-f472-4b3f-9f58-330dbef6d0e9 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:43.458536 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a93241-b453-493e-85b8-8dbcc8805247 None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:43.458746 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865ff85e-f472-4b3f-9f58-330dbef6d0e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:43.458840 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a93241-b453-493e-85b8-8dbcc8805247 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:43.459001 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865ff85e-f472-4b3f-9f58-330dbef6d0e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:43.459100 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a93241-b453-493e-85b8-8dbcc8805247 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:43.459740 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a93241-b453-493e-85b8-8dbcc8805247 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:43.461101 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a93241-b453-493e-85b8-8dbcc8805247 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:43.541084 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 414/826] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 15 19:15:43 2022] POST /identity/v3/endpoints => generated 327 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:15:43.542436 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48aeef8-1942-4625-967b-974e818d129b None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:43.542726 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48aeef8-1942-4625-967b-974e818d129b None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:43.542998 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48aeef8-1942-4625-967b-974e818d129b None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:43.544383 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 415/827] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:15:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:15:43.551418 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865ff85e-f472-4b3f-9f58-330dbef6d0e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:43.558041 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865ff85e-f472-4b3f-9f58-330dbef6d0e9 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:43.560547 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f62b1-cdc8-406d-94fc-8a0501fb95e7 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:43.560833 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f62b1-cdc8-406d-94fc-8a0501fb95e7 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:43.561256 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f62b1-cdc8-406d-94fc-8a0501fb95e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:43.597378 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-5a2f62b1-cdc8-406d-94fc-8a0501fb95e7 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:15:43.600501 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-5a2f62b1-cdc8-406d-94fc-8a0501fb95e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:15:43.726257 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:43.749973 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 416/828] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:15:43 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 19:15:43.751727 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 413/829] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 15 19:15:43 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 19:15:43.763383 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2f62b1-cdc8-406d-94fc-8a0501fb95e7 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:43.781357 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865ff85e-f472-4b3f-9f58-330dbef6d0e9 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:43.835125 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:43.878831 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2f62b1-cdc8-406d-94fc-8a0501fb95e7 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:43.892627 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865ff85e-f472-4b3f-9f58-330dbef6d0e9 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:43.920616 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24e1a35-6be9-4c23-9512-8bf02b28b296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:43.921522 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24e1a35-6be9-4c23-9512-8bf02b28b296 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:43.921881 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24e1a35-6be9-4c23-9512-8bf02b28b296 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:43.922163 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24e1a35-6be9-4c23-9512-8bf02b28b296 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:43.929889 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8246f8-f966-4472-a7dd-bd4b9402ec3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:43.930748 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8246f8-f966-4472-a7dd-bd4b9402ec3f None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:43.931031 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8246f8-f966-4472-a7dd-bd4b9402ec3f None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:43.931310 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8246f8-f966-4472-a7dd-bd4b9402ec3f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:43.934044 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24e1a35-6be9-4c23-9512-8bf02b28b296 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:43.935750 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24e1a35-6be9-4c23-9512-8bf02b28b296 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:43.942129 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8246f8-f966-4472-a7dd-bd4b9402ec3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:15:43.949717 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-5c8246f8-f966-4472-a7dd-bd4b9402ec3f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:15:43.949717 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:43.949717 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:43.949717 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:43.949717 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:43.949717 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:43.949717 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:43.949717 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:43.949717 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:43.949717 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:43.949717 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:43.949717 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:43.949717 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:15:43.949717 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:15:43.949717 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:15:43.949717 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:15:43.950696 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:15:43.950696 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:15:43.950696 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:15:43.950696 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:15:43.950696 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:43.950696 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:43.950696 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:43.950696 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:43.950696 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:43.950696 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:43.950696 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:43.950696 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:43.950696 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:43.950696 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:43.950696 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:43.950696 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:43.951538 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:43.951538 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:43.951538 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:43.951538 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:43.951538 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:15:43.951538 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:15:43.951538 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:15:43.951538 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:15:43.951538 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:15:43.951538 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:15:43.951538 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:15:43.951538 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:43.951538 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 414/830] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jul 15 19:15:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:15:43.958060 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8246f8-f966-4472-a7dd-bd4b9402ec3f None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:43.958244 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-a24e1a35-6be9-4c23-9512-8bf02b28b296 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:43.958244 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:43.958244 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:43.958244 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:43.958244 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:43.958244 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:43.958244 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:43.958244 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:43.958244 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:43.958244 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:43.958244 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:43.958244 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:43.958244 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:15:43.958244 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:15:43.958244 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:15:43.959330 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:15:43.959330 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:43.959330 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:43.959330 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:43.959330 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:43.959330 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:43.959330 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:43.959330 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:43.959330 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:43.959330 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:43.959330 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:43.959330 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:43.959330 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:43.959330 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:43.959330 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:43.959330 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:43.959330 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:43.960448 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:15:43.960448 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:15:43.960448 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:43.960448 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:43.962509 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 417/831] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:15:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:43.968179 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24e1a35-6be9-4c23-9512-8bf02b28b296 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:44.006513 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:44.076058 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8246f8-f966-4472-a7dd-bd4b9402ec3f None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:44.106961 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24e1a35-6be9-4c23-9512-8bf02b28b296 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:44.120177 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:44.121192 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74c86db-abf7-4a87-bf91-fb1d30bc3464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:44.122152 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74c86db-abf7-4a87-bf91-fb1d30bc3464 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:44.122641 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74c86db-abf7-4a87-bf91-fb1d30bc3464 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:44.123123 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74c86db-abf7-4a87-bf91-fb1d30bc3464 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:44.124072 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74c86db-abf7-4a87-bf91-fb1d30bc3464 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:44.124598 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74c86db-abf7-4a87-bf91-fb1d30bc3464 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:44.126280 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74c86db-abf7-4a87-bf91-fb1d30bc3464 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:44.136425 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 415/832] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 15 19:15:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:44.143425 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74c86db-abf7-4a87-bf91-fb1d30bc3464 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:44.146542 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520e96c0-e5b8-4cae-a77c-4cda76c36b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:44.147219 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520e96c0-e5b8-4cae-a77c-4cda76c36b98 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:44.147504 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520e96c0-e5b8-4cae-a77c-4cda76c36b98 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:44.147777 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520e96c0-e5b8-4cae-a77c-4cda76c36b98 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:44.148711 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520e96c0-e5b8-4cae-a77c-4cda76c36b98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:44.149024 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520e96c0-e5b8-4cae-a77c-4cda76c36b98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:44.152144 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520e96c0-e5b8-4cae-a77c-4cda76c36b98 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:44.167280 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 418/833] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:15:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:15:44.172034 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520e96c0-e5b8-4cae-a77c-4cda76c36b98 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:44.272472 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:44.276104 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74c86db-abf7-4a87-bf91-fb1d30bc3464 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:44.280507 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520e96c0-e5b8-4cae-a77c-4cda76c36b98 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:44.312855 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54688746-b49b-4b03-944e-58b1d352ab24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:44.313657 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54688746-b49b-4b03-944e-58b1d352ab24 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:44.313992 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54688746-b49b-4b03-944e-58b1d352ab24 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:44.314279 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54688746-b49b-4b03-944e-58b1d352ab24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:44.318942 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf4ea28-4180-4e74-90bf-16ba477319ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:44.320078 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf4ea28-4180-4e74-90bf-16ba477319ce None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:44.320655 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf4ea28-4180-4e74-90bf-16ba477319ce None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:44.321126 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf4ea28-4180-4e74-90bf-16ba477319ce None admin] PATH_INFO: `/v3/projects/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:44.327996 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54688746-b49b-4b03-944e-58b1d352ab24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:44.328974 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf4ea28-4180-4e74-90bf-16ba477319ce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:44.329483 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54688746-b49b-4b03-944e-58b1d352ab24 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:44.331038 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf4ea28-4180-4e74-90bf-16ba477319ce None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:44.346403 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-7cf4ea28-4180-4e74-90bf-16ba477319ce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:15:44.346403 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:44.346403 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:44.346403 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:44.346403 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:44.346403 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:44.346403 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:44.346403 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:44.346403 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:44.346403 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:44.346403 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:44.346403 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:44.346403 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 19:15:44.346403 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 19:15:44.346403 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 19:15:44.347318 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 19:15:44.347318 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:44.347318 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:44.347318 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:44.347318 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:44.347318 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:44.347318 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:44.347318 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:44.347318 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:44.347318 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:44.347318 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:44.347318 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:44.347318 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:44.347318 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:44.347318 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:44.347318 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:44.347318 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:44.348943 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 19:15:44.348943 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 19:15:44.348943 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:15:44.348943 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:15:44.348943 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:15:44.348943 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:15:44.348943 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:15:44.348943 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:44.349413 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-54688746-b49b-4b03-944e-58b1d352ab24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:44.349413 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:44.349413 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:44.349413 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:44.349413 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:44.349413 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:44.349413 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:44.349413 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:44.349413 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:44.349413 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:44.349413 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:44.349413 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:44.349413 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:15:44.349413 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:15:44.349413 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:15:44.350864 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:15:44.350864 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:44.350864 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:44.350864 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:44.350864 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:44.350864 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:44.350864 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:44.350864 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:44.350864 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:44.350864 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:44.350864 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:44.350864 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:44.350864 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:44.350864 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:44.350864 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:44.350864 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:44.350864 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:44.351753 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:15:44.351753 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:15:44.351753 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:44.351753 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:44.353898 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 419/834] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 15 19:15:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:15:44.357904 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 416/835] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:15:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:15:44.359158 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf4ea28-4180-4e74-90bf-16ba477319ce None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:44.360396 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54688746-b49b-4b03-944e-58b1d352ab24 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:44.399550 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:44.490957 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54688746-b49b-4b03-944e-58b1d352ab24 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:44.503335 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf4ea28-4180-4e74-90bf-16ba477319ce None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:44.527449 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fbdbf8-5a04-4134-9266-9aadf06daf03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:44.528126 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fbdbf8-5a04-4134-9266-9aadf06daf03 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:44.528479 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fbdbf8-5a04-4134-9266-9aadf06daf03 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:44.528826 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fbdbf8-5a04-4134-9266-9aadf06daf03 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:44.529631 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fbdbf8-5a04-4134-9266-9aadf06daf03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:44.530034 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fbdbf8-5a04-4134-9266-9aadf06daf03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:44.531512 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fbdbf8-5a04-4134-9266-9aadf06daf03 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:44.550125 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 417/836] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:15:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:44.551847 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fbdbf8-5a04-4134-9266-9aadf06daf03 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:44.555953 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c7fa18-38a9-469e-b4bf-880f84114871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:44.556689 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c7fa18-38a9-469e-b4bf-880f84114871 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:44.557726 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c7fa18-38a9-469e-b4bf-880f84114871 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:44.557726 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c7fa18-38a9-469e-b4bf-880f84114871 None admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:44.558065 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c7fa18-38a9-469e-b4bf-880f84114871 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:44.558376 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c7fa18-38a9-469e-b4bf-880f84114871 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:44.559805 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c7fa18-38a9-469e-b4bf-880f84114871 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:44.570625 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:44.573798 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 420/837] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jul 15 19:15:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:15:44.579477 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c7fa18-38a9-469e-b4bf-880f84114871 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:44.671636 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fbdbf8-5a04-4134-9266-9aadf06daf03 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:44.687666 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:44.698543 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c7fa18-38a9-469e-b4bf-880f84114871 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:44.708101 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63f592a-829e-476d-82d7-b9923ccd6367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:44.709030 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63f592a-829e-476d-82d7-b9923ccd6367 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:44.709275 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63f592a-829e-476d-82d7-b9923ccd6367 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:44.709633 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63f592a-829e-476d-82d7-b9923ccd6367 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:44.718851 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63f592a-829e-476d-82d7-b9923ccd6367 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:44.720285 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63f592a-829e-476d-82d7-b9923ccd6367 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:44.734260 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3d1fe8-38f0-4ae5-bc8f-1d4175b5f996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:44.734984 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3d1fe8-38f0-4ae5-bc8f-1d4175b5f996 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:44.735273 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3d1fe8-38f0-4ae5-bc8f-1d4175b5f996 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:44.735502 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-d63f592a-829e-476d-82d7-b9923ccd6367 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 15 19:15:44.735502 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:44.735502 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:44.735502 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:44.735502 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:44.735502 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:44.735502 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:44.735502 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:44.735502 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:44.735502 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:44.735502 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:44.735502 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:44.735502 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 19:15:44.735502 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 19:15:44.735502 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 19:15:44.735502 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 19:15:44.736387 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:44.736387 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:44.736387 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:15:44.736387 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:15:44.736387 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:15:44.736387 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:15:44.736387 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:44.736387 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:44.736387 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:44.736387 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:44.736387 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:44.736387 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:44.736387 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:44.736387 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:44.736387 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:44.736387 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:44.736387 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:44.736387 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:44.737325 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:44.737325 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:44.737325 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 19:15:44.737325 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 19:15:44.737325 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 19:15:44.737325 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 19:15:44.737325 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 19:15:44.737325 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 19:15:44.737325 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 15 19:15:44.737325 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:44.738356 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3d1fe8-38f0-4ae5-bc8f-1d4175b5f996 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:44.738356 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3d1fe8-38f0-4ae5-bc8f-1d4175b5f996 None admin] RBAC: Adding query filter params (role.id=4318f86ba9f74cd4a0bef1c311276657, scope.project.id=55a756ccaef14cb49b26b0d68dd0b466, user.id=7673a3b5c0324e07a9535ac70ed7a491) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:44.738993 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 418/838] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jul 15 19:15:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:44.738993 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3624119c-b1cb-47e7-a011-d2fea959277a None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:44.739118 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3624119c-b1cb-47e7-a011-d2fea959277a None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:44.739382 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3624119c-b1cb-47e7-a011-d2fea959277a None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:44.742047 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3d1fe8-38f0-4ae5-bc8f-1d4175b5f996 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:44.743358 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3d1fe8-38f0-4ae5-bc8f-1d4175b5f996 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:44.744807 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 419/839] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:15:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:15:44.745622 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3624119c-b1cb-47e7-a011-d2fea959277a None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:44.758188 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 421/840] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 15 19:15:44 2022] GET /identity/v3/role_assignments?user.id=7673a3b5c0324e07a9535ac70ed7a491&scope.project.id=55a756ccaef14cb49b26b0d68dd0b466&role.id=4318f86ba9f74cd4a0bef1c311276657 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:15:44.760189 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d246e3a-a37f-422c-9add-50c73265b73e None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:44.760482 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d246e3a-a37f-422c-9add-50c73265b73e None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:44.760760 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d246e3a-a37f-422c-9add-50c73265b73e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:44.779444 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-6d246e3a-a37f-422c-9add-50c73265b73e None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:15:44.781477 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-6d246e3a-a37f-422c-9add-50c73265b73e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:15:44.848702 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:44.863242 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 422/841] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:15:44 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:15:44.887597 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d246e3a-a37f-422c-9add-50c73265b73e None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:44.897883 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3624119c-b1cb-47e7-a011-d2fea959277a None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:44.937701 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11698ea7-5c31-4481-afed-bd7094062e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:44.938781 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11698ea7-5c31-4481-afed-bd7094062e2e None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:44.939282 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11698ea7-5c31-4481-afed-bd7094062e2e None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:44.939759 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11698ea7-5c31-4481-afed-bd7094062e2e None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:44.940807 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11698ea7-5c31-4481-afed-bd7094062e2e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:44.941318 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11698ea7-5c31-4481-afed-bd7094062e2e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:44.943048 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11698ea7-5c31-4481-afed-bd7094062e2e None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:44.962220 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 420/842] 10.222.0.33 () {56 vars in 1135 bytes} [Fri Jul 15 19:15:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:44.974676 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11698ea7-5c31-4481-afed-bd7094062e2e None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:44.984917 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:45.005269 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d246e3a-a37f-422c-9add-50c73265b73e None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:45.045262 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a1c978-703a-4630-9617-1dfb1afbe2a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:45.046563 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a1c978-703a-4630-9617-1dfb1afbe2a3 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:45.047055 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a1c978-703a-4630-9617-1dfb1afbe2a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:45.047531 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a1c978-703a-4630-9617-1dfb1afbe2a3 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:45.048388 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a1c978-703a-4630-9617-1dfb1afbe2a3 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:45.050011 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a1c978-703a-4630-9617-1dfb1afbe2a3 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:45.062944 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-77a1c978-703a-4630-9617-1dfb1afbe2a3 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 15 19:15:45.062944 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:45.062944 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 19:15:45.062944 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 19:15:45.062944 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 19:15:45.062944 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 19:15:45.062944 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 19:15:45.062944 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:15:45.062944 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 15 19:15:45.062944 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:45.062944 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 19:15:45.062944 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:45.062944 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:45.062944 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:45.062944 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:45.062944 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:45.062944 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:45.063993 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:45.063993 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:45.063993 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:45.063993 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:45.063993 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:45.063993 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:45.063993 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 19:15:45.063993 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 19:15:45.063993 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 19:15:45.063993 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 19:15:45.063993 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:45.063993 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:45.063993 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:45.063993 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:45.063993 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:45.063993 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:45.064836 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:45.064836 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:45.064836 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:45.064836 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:45.064836 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:45.064836 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:45.064836 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:45.064836 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:45.064836 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:45.064836 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:45.064836 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 19:15:45.064836 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:15:45.064836 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 15 19:15:45.064836 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:45.067776 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 423/843] 10.222.0.33 () {56 vars in 1114 bytes} [Fri Jul 15 19:15:44 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:15:45.075174 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a1c978-703a-4630-9617-1dfb1afbe2a3 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:45.100357 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11698ea7-5c31-4481-afed-bd7094062e2e None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:45.200192 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a1c978-703a-4630-9617-1dfb1afbe2a3 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:45.240635 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a5254e-ab16-406e-b7c4-dc659bf0eb50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:45.241307 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a5254e-ab16-406e-b7c4-dc659bf0eb50 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:45.241585 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a5254e-ab16-406e-b7c4-dc659bf0eb50 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:45.241886 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a5254e-ab16-406e-b7c4-dc659bf0eb50 None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:45.242600 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a5254e-ab16-406e-b7c4-dc659bf0eb50 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:45.242902 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a5254e-ab16-406e-b7c4-dc659bf0eb50 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:45.244260 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a5254e-ab16-406e-b7c4-dc659bf0eb50 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:45.253852 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 424/844] 10.222.0.33 () {56 vars in 1114 bytes} [Fri Jul 15 19:15:45 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:45.258396 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a5254e-ab16-406e-b7c4-dc659bf0eb50 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:45.260526 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:45.266787 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f880a2-4b80-4f62-948e-ff2951868623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:45.267473 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f880a2-4b80-4f62-948e-ff2951868623 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:45.267754 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f880a2-4b80-4f62-948e-ff2951868623 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:45.268076 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f880a2-4b80-4f62-948e-ff2951868623 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:45.278719 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:45.343769 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f880a2-4b80-4f62-948e-ff2951868623 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:45.345232 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f880a2-4b80-4f62-948e-ff2951868623 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:45.389088 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a5254e-ab16-406e-b7c4-dc659bf0eb50 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:45.434011 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286fe586-4e6b-4fff-9a39-3339bd9b544d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:45.435329 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286fe586-4e6b-4fff-9a39-3339bd9b544d None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:45.435815 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286fe586-4e6b-4fff-9a39-3339bd9b544d None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:45.436288 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286fe586-4e6b-4fff-9a39-3339bd9b544d None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:45.437601 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286fe586-4e6b-4fff-9a39-3339bd9b544d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:45.438138 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286fe586-4e6b-4fff-9a39-3339bd9b544d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:45.440145 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286fe586-4e6b-4fff-9a39-3339bd9b544d None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:45.453791 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 425/845] 10.222.0.33 () {56 vars in 1114 bytes} [Fri Jul 15 19:15:45 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:45.460407 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 421/846] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 15 19:15:44 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 19:15:45.465014 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e989e4e4-edf6-4e66-a846-40674966e7c2 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:45.465297 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e989e4e4-edf6-4e66-a846-40674966e7c2 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:45.465571 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e989e4e4-edf6-4e66-a846-40674966e7c2 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:45.467191 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 426/847] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:15:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:15:45.471296 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f880a2-4b80-4f62-948e-ff2951868623 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:45.472626 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8115a2e8-a245-4eb3-b59c-60b1a7f5b325 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:45.472993 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8115a2e8-a245-4eb3-b59c-60b1a7f5b325 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:45.473277 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8115a2e8-a245-4eb3-b59c-60b1a7f5b325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:45.493669 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-8115a2e8-a245-4eb3-b59c-60b1a7f5b325 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:15:45.495266 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-8115a2e8-a245-4eb3-b59c-60b1a7f5b325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:15:45.515174 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:45.579770 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 427/848] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:15:45 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 19:15:45.607752 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8115a2e8-a245-4eb3-b59c-60b1a7f5b325 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:45.615258 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f880a2-4b80-4f62-948e-ff2951868623 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:45.616361 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:45.654990 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e447483-c596-42e8-b253-ca5c1df9212d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:45.656130 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e447483-c596-42e8-b253-ca5c1df9212d None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:45.656412 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e447483-c596-42e8-b253-ca5c1df9212d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:45.657003 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e447483-c596-42e8-b253-ca5c1df9212d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:45.672535 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e447483-c596-42e8-b253-ca5c1df9212d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:45.672535 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e447483-c596-42e8-b253-ca5c1df9212d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:45.695486 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-5e447483-c596-42e8-b253-ca5c1df9212d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:45.695486 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:45.695486 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:45.695486 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:45.695486 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:45.695486 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:45.695486 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:45.695486 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:45.695486 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:45.695486 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:45.695486 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:45.695486 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:45.695486 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:15:45.695486 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:15:45.695486 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:15:45.696431 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:15:45.696431 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:45.696431 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:45.696431 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:45.696431 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:45.696431 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:45.696431 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:45.696431 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:45.696431 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:45.696431 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:45.696431 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:45.696431 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:45.696431 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:45.696431 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:45.696431 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:45.696431 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:45.696431 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:45.697331 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:15:45.697331 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:15:45.697331 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:45.697331 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:45.697331 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 422/849] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:15:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:45.702415 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e447483-c596-42e8-b253-ca5c1df9212d None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:45.726410 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8115a2e8-a245-4eb3-b59c-60b1a7f5b325 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:45.764547 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779c47da-49dc-4b11-8f17-cab2fa01f1e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:45.765345 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779c47da-49dc-4b11-8f17-cab2fa01f1e5 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:45.765638 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779c47da-49dc-4b11-8f17-cab2fa01f1e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:45.766239 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779c47da-49dc-4b11-8f17-cab2fa01f1e5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:45.783523 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779c47da-49dc-4b11-8f17-cab2fa01f1e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:15:45.792127 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-779c47da-49dc-4b11-8f17-cab2fa01f1e5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:15:45.792127 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:45.792127 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:45.792127 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:45.792127 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:45.792127 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:45.792127 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:45.792127 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:45.792127 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:45.792127 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:45.792127 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:45.792127 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:45.792127 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:15:45.792127 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:15:45.792127 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:15:45.792127 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:15:45.793070 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:15:45.793070 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:15:45.793070 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:15:45.793070 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:15:45.793070 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:45.793070 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:45.793070 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:45.793070 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:45.793070 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:45.793070 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:45.793070 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:45.793070 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:45.793070 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:45.793070 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:45.793070 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:45.793070 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:45.793945 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:45.793945 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:45.793945 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:45.793945 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:45.793945 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:15:45.793945 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:15:45.793945 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:15:45.793945 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:15:45.793945 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:15:45.793945 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:15:45.793945 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:15:45.793945 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:45.793945 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 428/850] 10.222.0.33 () {56 vars in 1102 bytes} [Fri Jul 15 19:15:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:15:45.804591 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779c47da-49dc-4b11-8f17-cab2fa01f1e5 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:45.817331 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:45.841355 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e447483-c596-42e8-b253-ca5c1df9212d None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:45.883509 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:45.888715 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d2dc7d-e2fb-4c18-972d-bbe288ec57e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:45.890308 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d2dc7d-e2fb-4c18-972d-bbe288ec57e6 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:45.890589 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d2dc7d-e2fb-4c18-972d-bbe288ec57e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:45.890918 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d2dc7d-e2fb-4c18-972d-bbe288ec57e6 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:45.891712 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d2dc7d-e2fb-4c18-972d-bbe288ec57e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:45.892013 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d2dc7d-e2fb-4c18-972d-bbe288ec57e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:45.893389 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d2dc7d-e2fb-4c18-972d-bbe288ec57e6 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:45.906801 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 423/851] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:15:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:45.912469 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d2dc7d-e2fb-4c18-972d-bbe288ec57e6 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:45.932074 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779c47da-49dc-4b11-8f17-cab2fa01f1e5 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:45.980302 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4861a6-f520-4782-9bcd-348b643c6ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:45.980987 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4861a6-f520-4782-9bcd-348b643c6ac3 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:45.981268 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4861a6-f520-4782-9bcd-348b643c6ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:45.981541 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4861a6-f520-4782-9bcd-348b643c6ac3 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:45.982507 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4861a6-f520-4782-9bcd-348b643c6ac3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:45.982633 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4861a6-f520-4782-9bcd-348b643c6ac3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:45.984034 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4861a6-f520-4782-9bcd-348b643c6ac3 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:45.994671 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 429/852] 10.222.0.33 () {56 vars in 1103 bytes} [Fri Jul 15 19:15:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:46.000050 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4861a6-f520-4782-9bcd-348b643c6ac3 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:46.051590 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d2dc7d-e2fb-4c18-972d-bbe288ec57e6 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:46.100144 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed40039-cf5b-4dcf-a041-fa20fea02821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:46.101319 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed40039-cf5b-4dcf-a041-fa20fea02821 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:46.101851 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed40039-cf5b-4dcf-a041-fa20fea02821 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:46.102336 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed40039-cf5b-4dcf-a041-fa20fea02821 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:46.112525 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed40039-cf5b-4dcf-a041-fa20fea02821 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:46.114973 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed40039-cf5b-4dcf-a041-fa20fea02821 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:46.115796 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4861a6-f520-4782-9bcd-348b643c6ac3 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:46.135562 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-0ed40039-cf5b-4dcf-a041-fa20fea02821 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:15:46.135562 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:46.135562 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:46.135562 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:46.135562 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:46.135562 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:46.135562 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:46.135562 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:46.135562 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:46.135562 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:46.135562 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:46.135562 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:46.135562 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 19:15:46.135562 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 19:15:46.135562 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 19:15:46.136481 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 19:15:46.136481 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:46.136481 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:46.136481 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:46.136481 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:46.136481 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:46.136481 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:46.136481 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:46.136481 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:46.136481 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:46.136481 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:46.136481 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:46.136481 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:46.136481 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:46.136481 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:46.136481 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:46.136481 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:46.137396 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 19:15:46.137396 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 19:15:46.137396 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:15:46.137396 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:15:46.137396 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:15:46.137396 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:15:46.137396 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:15:46.137396 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:46.143799 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 424/853] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 15 19:15:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:46.146662 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed40039-cf5b-4dcf-a041-fa20fea02821 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:46.150716 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:46.152695 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:46.160723 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec41db2-87bc-4522-86e6-7d551bf9192e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:46.161592 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec41db2-87bc-4522-86e6-7d551bf9192e None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:46.166457 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec41db2-87bc-4522-86e6-7d551bf9192e None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:46.166457 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec41db2-87bc-4522-86e6-7d551bf9192e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:46.178355 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec41db2-87bc-4522-86e6-7d551bf9192e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:46.179919 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec41db2-87bc-4522-86e6-7d551bf9192e None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:46.199676 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-aec41db2-87bc-4522-86e6-7d551bf9192e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:46.199676 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:46.199676 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:46.199676 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:46.199676 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:46.199676 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:46.199676 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:46.199676 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:46.199676 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:46.199676 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:46.199676 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:46.199676 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:46.199676 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:15:46.199676 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:15:46.199676 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:15:46.200617 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:15:46.200617 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:46.200617 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:46.200617 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:46.200617 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:46.200617 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:46.200617 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:46.200617 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:46.200617 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:46.200617 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:46.200617 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:46.200617 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:46.200617 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:46.200617 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:46.200617 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:46.200617 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:46.200617 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:46.201518 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:15:46.201518 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:15:46.201518 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:46.201518 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:46.205880 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 430/854] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:15:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:15:46.207083 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec41db2-87bc-4522-86e6-7d551bf9192e None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:46.286847 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed40039-cf5b-4dcf-a041-fa20fea02821 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:46.330010 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec41db2-87bc-4522-86e6-7d551bf9192e None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:46.331054 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154683a7-77d9-4a4d-ac2c-68846ad2b206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:46.331792 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154683a7-77d9-4a4d-ac2c-68846ad2b206 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:46.332173 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154683a7-77d9-4a4d-ac2c-68846ad2b206 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:46.332727 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154683a7-77d9-4a4d-ac2c-68846ad2b206 None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:46.333497 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154683a7-77d9-4a4d-ac2c-68846ad2b206 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:46.334628 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154683a7-77d9-4a4d-ac2c-68846ad2b206 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:46.335410 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154683a7-77d9-4a4d-ac2c-68846ad2b206 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:46.353043 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 425/855] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jul 15 19:15:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:46.355208 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154683a7-77d9-4a4d-ac2c-68846ad2b206 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:46.376650 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbb34f0-cc22-414e-8932-f95e40b2f7da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:46.377340 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbb34f0-cc22-414e-8932-f95e40b2f7da None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:46.377620 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbb34f0-cc22-414e-8932-f95e40b2f7da None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:46.377925 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbb34f0-cc22-414e-8932-f95e40b2f7da None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:46.378723 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbb34f0-cc22-414e-8932-f95e40b2f7da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:46.379580 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbb34f0-cc22-414e-8932-f95e40b2f7da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:46.381000 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbb34f0-cc22-414e-8932-f95e40b2f7da None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:46.398155 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 431/856] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:15:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:46.399868 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbb34f0-cc22-414e-8932-f95e40b2f7da None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:46.437240 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:46.443823 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:46.487287 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154683a7-77d9-4a4d-ac2c-68846ad2b206 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:46.508030 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbb34f0-cc22-414e-8932-f95e40b2f7da None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:46.533935 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476e28e0-72a3-4ac2-9004-93aff3016712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:46.536073 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e28e0-72a3-4ac2-9004-93aff3016712 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:46.536512 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e28e0-72a3-4ac2-9004-93aff3016712 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:46.537078 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e28e0-72a3-4ac2-9004-93aff3016712 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:46.538426 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476e28e0-72a3-4ac2-9004-93aff3016712 None admin] RBAC: Adding query filter params (role.id=4318f86ba9f74cd4a0bef1c311276657, scope.project.id=55a756ccaef14cb49b26b0d68dd0b466, user.id=ef76f7fe39a3499aaeebf85c176804f3) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:46.539139 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476e28e0-72a3-4ac2-9004-93aff3016712 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:46.541169 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476e28e0-72a3-4ac2-9004-93aff3016712 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:46.546109 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ac5dc3-64d5-4e33-9630-e0607219cf25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:46.547109 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ac5dc3-64d5-4e33-9630-e0607219cf25 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:46.547489 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ac5dc3-64d5-4e33-9630-e0607219cf25 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:46.547862 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ac5dc3-64d5-4e33-9630-e0607219cf25 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:46.558034 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 426/857] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 15 19:15:46 2022] GET /identity/v3/role_assignments?user.id=ef76f7fe39a3499aaeebf85c176804f3&scope.project.id=55a756ccaef14cb49b26b0d68dd0b466&role.id=4318f86ba9f74cd4a0bef1c311276657 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:46.564490 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ac5dc3-64d5-4e33-9630-e0607219cf25 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:46.566090 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ac5dc3-64d5-4e33-9630-e0607219cf25 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:46.583983 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-36ac5dc3-64d5-4e33-9630-e0607219cf25 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 15 19:15:46.583983 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:46.583983 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:46.583983 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:46.583983 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:46.583983 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:46.583983 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:46.583983 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:46.583983 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:46.583983 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:46.583983 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:46.583983 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:46.583983 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 19:15:46.583983 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 19:15:46.583983 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 19:15:46.583983 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 19:15:46.585000 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:46.585000 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:46.585000 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:15:46.585000 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:15:46.585000 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:15:46.585000 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:15:46.585000 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:46.585000 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:46.585000 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:46.585000 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:46.585000 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:46.585000 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:46.585000 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:46.585000 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:46.585000 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:46.585000 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:46.585000 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:46.585000 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:46.586380 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:46.586380 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:46.586380 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 19:15:46.586380 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 19:15:46.586380 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 19:15:46.586380 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 19:15:46.586380 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 19:15:46.586380 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 19:15:46.586380 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 15 19:15:46.586380 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:46.586380 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 432/858] 10.222.0.33 () {56 vars in 1117 bytes} [Fri Jul 15 19:15:46 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:15:46.591484 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476e28e0-72a3-4ac2-9004-93aff3016712 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:46.704789 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476e28e0-72a3-4ac2-9004-93aff3016712 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:46.733711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:46.743640 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdcbd7b-51af-4e6a-8fd5-9bf4ae1e55ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:46.744455 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdcbd7b-51af-4e6a-8fd5-9bf4ae1e55ed None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:46.744737 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdcbd7b-51af-4e6a-8fd5-9bf4ae1e55ed None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:46.748961 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdcbd7b-51af-4e6a-8fd5-9bf4ae1e55ed None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:46.748961 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdcbd7b-51af-4e6a-8fd5-9bf4ae1e55ed None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:46.748961 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdcbd7b-51af-4e6a-8fd5-9bf4ae1e55ed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:46.748961 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdcbd7b-51af-4e6a-8fd5-9bf4ae1e55ed None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:46.766402 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 427/859] 10.222.0.33 () {56 vars in 1149 bytes} [Fri Jul 15 19:15:46 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:46.776340 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ac5dc3-64d5-4e33-9630-e0607219cf25 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:46.899236 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2dfc4a-2b06-417f-921d-c3b3735be9a3 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:46.899566 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2dfc4a-2b06-417f-921d-c3b3735be9a3 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:46.899799 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2dfc4a-2b06-417f-921d-c3b3735be9a3 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:46.901750 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 428/860] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:15:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:15:46.905898 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:46.907439 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec10f54-681f-4421-960c-81b12646d465 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:46.907733 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec10f54-681f-4421-960c-81b12646d465 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:46.908013 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec10f54-681f-4421-960c-81b12646d465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:46.909831 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ac5dc3-64d5-4e33-9630-e0607219cf25 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:46.929595 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-6ec10f54-681f-4421-960c-81b12646d465 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:15:46.930848 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-6ec10f54-681f-4421-960c-81b12646d465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:15:46.948096 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c0957b-923a-4daa-b5e8-f5c87c945cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:46.948745 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c0957b-923a-4daa-b5e8-f5c87c945cf3 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:46.949025 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c0957b-923a-4daa-b5e8-f5c87c945cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:46.949296 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c0957b-923a-4daa-b5e8-f5c87c945cf3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:47.018125 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 429/861] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:15:46 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:15:47.035160 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec10f54-681f-4421-960c-81b12646d465 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:47.035862 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c0957b-923a-4daa-b5e8-f5c87c945cf3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:47.037135 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c0957b-923a-4daa-b5e8-f5c87c945cf3 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:47.149890 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec10f54-681f-4421-960c-81b12646d465 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:47.166818 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:47.173545 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:47.189871 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 433/862] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 15 19:15:46 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 19:15:47.196785 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c0957b-923a-4daa-b5e8-f5c87c945cf3 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:47.208387 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c90f137-058b-45b8-b93c-d8b4a6e90223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:47.210059 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c90f137-058b-45b8-b93c-d8b4a6e90223 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:47.210828 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c90f137-058b-45b8-b93c-d8b4a6e90223 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:47.211326 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c90f137-058b-45b8-b93c-d8b4a6e90223 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:47.212447 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c90f137-058b-45b8-b93c-d8b4a6e90223 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:47.214705 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c90f137-058b-45b8-b93c-d8b4a6e90223 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:47.231282 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-9c90f137-058b-45b8-b93c-d8b4a6e90223 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 15 19:15:47.231282 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:47.231282 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 19:15:47.231282 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 19:15:47.231282 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 19:15:47.231282 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 19:15:47.231282 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 19:15:47.231282 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:15:47.231282 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 15 19:15:47.231282 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:47.231282 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 19:15:47.231282 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:47.231282 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:47.231282 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:47.231282 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:47.231282 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:47.232264 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:47.232264 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:47.232264 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:47.232264 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:47.232264 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:47.232264 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:47.232264 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:47.232264 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 19:15:47.232264 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 19:15:47.232264 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 19:15:47.232264 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 19:15:47.232264 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:47.232264 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:47.232264 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:47.232264 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:47.232264 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:47.233127 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:47.233127 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:47.233127 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:47.233127 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:47.233127 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:47.233127 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:47.233127 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:47.233127 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:47.233127 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:47.233127 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:47.233127 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:47.233127 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 19:15:47.233127 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:15:47.233127 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 15 19:15:47.233127 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:47.240813 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 430/863] 10.222.0.33 () {56 vars in 1132 bytes} [Fri Jul 15 19:15:47 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:15:47.241913 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c90f137-058b-45b8-b93c-d8b4a6e90223 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:47.308615 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c0957b-923a-4daa-b5e8-f5c87c945cf3 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:47.347813 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c90f137-058b-45b8-b93c-d8b4a6e90223 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:47.348044 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daf5215-b832-47d2-91bc-d62caa07ff1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:47.348884 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daf5215-b832-47d2-91bc-d62caa07ff1e None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:47.349264 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daf5215-b832-47d2-91bc-d62caa07ff1e None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:47.349639 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daf5215-b832-47d2-91bc-d62caa07ff1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:47.361145 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daf5215-b832-47d2-91bc-d62caa07ff1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:47.366801 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daf5215-b832-47d2-91bc-d62caa07ff1e None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:47.384082 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-1daf5215-b832-47d2-91bc-d62caa07ff1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:47.384082 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:47.384082 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:47.384082 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:47.384082 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:47.384082 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:47.384082 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:47.384082 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:47.384082 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:47.384082 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:47.384082 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:47.384082 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:47.384082 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:15:47.384082 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:15:47.384082 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:15:47.385133 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:15:47.385133 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:47.385133 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:47.385133 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:47.385133 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:47.385133 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:47.385133 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:47.385133 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:47.385133 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:47.385133 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:47.385133 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:47.385133 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:47.385133 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:47.385133 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:47.385133 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:47.385133 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:47.385133 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:47.386073 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:15:47.386073 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:15:47.386073 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:47.386073 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:47.387686 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fee4c27-2bc6-4713-b212-a6916f3fbf82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:47.388548 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee4c27-2bc6-4713-b212-a6916f3fbf82 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:47.388991 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee4c27-2bc6-4713-b212-a6916f3fbf82 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:47.389443 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee4c27-2bc6-4713-b212-a6916f3fbf82 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:47.390361 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 434/864] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:15:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:15:47.390471 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fee4c27-2bc6-4713-b212-a6916f3fbf82 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:47.391232 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fee4c27-2bc6-4713-b212-a6916f3fbf82 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:47.392060 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daf5215-b832-47d2-91bc-d62caa07ff1e None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:47.392997 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fee4c27-2bc6-4713-b212-a6916f3fbf82 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:47.403406 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 431/865] 10.222.0.33 () {56 vars in 1126 bytes} [Fri Jul 15 19:15:47 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:47.404559 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fee4c27-2bc6-4713-b212-a6916f3fbf82 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:47.449675 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:47.453418 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:47.504633 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daf5215-b832-47d2-91bc-d62caa07ff1e None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:47.508100 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fee4c27-2bc6-4713-b212-a6916f3fbf82 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:47.542906 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02fd4de-683c-4828-9447-c3845cba5bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:47.543748 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971d934b-f4cf-4eef-8729-890c24c33268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:47.544514 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971d934b-f4cf-4eef-8729-890c24c33268 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:47.544797 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971d934b-f4cf-4eef-8729-890c24c33268 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:47.545358 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971d934b-f4cf-4eef-8729-890c24c33268 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:47.545702 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02fd4de-683c-4828-9447-c3845cba5bf0 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:47.546133 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971d934b-f4cf-4eef-8729-890c24c33268 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:47.546445 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971d934b-f4cf-4eef-8729-890c24c33268 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:47.546705 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02fd4de-683c-4828-9447-c3845cba5bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:47.546988 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02fd4de-683c-4828-9447-c3845cba5bf0 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:47.547716 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02fd4de-683c-4828-9447-c3845cba5bf0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:47.547821 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971d934b-f4cf-4eef-8729-890c24c33268 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:47.548024 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02fd4de-683c-4828-9447-c3845cba5bf0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:47.549374 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02fd4de-683c-4828-9447-c3845cba5bf0 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:47.553911 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 432/866] 10.222.0.33 () {56 vars in 1126 bytes} [Fri Jul 15 19:15:47 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:47.555517 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081ffed5-007c-4e1b-a85f-ae3e5ba3f23d None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:47.556110 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081ffed5-007c-4e1b-a85f-ae3e5ba3f23d None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:47.556587 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081ffed5-007c-4e1b-a85f-ae3e5ba3f23d None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:47.562689 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 433/867] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:15:47 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:15:47.564486 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8555bd6-9e95-4de2-889a-a3de1cc1bd59 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:47.564984 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8555bd6-9e95-4de2-889a-a3de1cc1bd59 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:47.565455 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8555bd6-9e95-4de2-889a-a3de1cc1bd59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:47.566615 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 435/868] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:15:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:15:47.567610 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02fd4de-683c-4828-9447-c3845cba5bf0 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:47.585349 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-d8555bd6-9e95-4de2-889a-a3de1cc1bd59 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:15:47.587158 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d8555bd6-9e95-4de2-889a-a3de1cc1bd59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:15:47.680156 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 434/869] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:15:47 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:15:47.688167 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02fd4de-683c-4828-9447-c3845cba5bf0 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:47.701503 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8555bd6-9e95-4de2-889a-a3de1cc1bd59 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:47.708015 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:47.731968 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ad103a-e981-4cbc-9acc-78f05ad60e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:47.733305 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ad103a-e981-4cbc-9acc-78f05ad60e4c None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:47.733598 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ad103a-e981-4cbc-9acc-78f05ad60e4c None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:47.734062 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ad103a-e981-4cbc-9acc-78f05ad60e4c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:47.744649 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ad103a-e981-4cbc-9acc-78f05ad60e4c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:47.746136 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ad103a-e981-4cbc-9acc-78f05ad60e4c None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:47.751058 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:47.761503 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-44ad103a-e981-4cbc-9acc-78f05ad60e4c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:15:47.761503 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:47.761503 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:47.761503 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:47.761503 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:47.761503 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:47.761503 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:47.761503 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:47.761503 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:47.761503 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:47.761503 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:47.761503 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:47.761503 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 19:15:47.761503 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 19:15:47.761503 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 19:15:47.765486 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 19:15:47.765486 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:47.765486 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:47.765486 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:47.765486 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:47.765486 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:47.765486 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:47.765486 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:47.765486 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:47.765486 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:47.765486 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:47.765486 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:47.765486 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:47.765486 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:47.765486 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:47.765486 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:47.765486 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:47.766486 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 19:15:47.766486 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 19:15:47.766486 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:15:47.766486 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:15:47.766486 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:15:47.766486 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:15:47.766486 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:15:47.766486 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:47.768034 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 436/870] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 15 19:15:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:47.768586 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ad103a-e981-4cbc-9acc-78f05ad60e4c None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:47.840712 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8555bd6-9e95-4de2-889a-a3de1cc1bd59 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:47.883230 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb44cc85-76c5-40e6-b52f-a19ef815e2d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:47.883952 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb44cc85-76c5-40e6-b52f-a19ef815e2d6 None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:47.884238 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb44cc85-76c5-40e6-b52f-a19ef815e2d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:47.884511 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb44cc85-76c5-40e6-b52f-a19ef815e2d6 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:47.885356 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb44cc85-76c5-40e6-b52f-a19ef815e2d6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:47.886978 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb44cc85-76c5-40e6-b52f-a19ef815e2d6 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:47.895590 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ad103a-e981-4cbc-9acc-78f05ad60e4c None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:47.933230 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a22f98-5169-4d8d-8f58-ffada51947c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:47.933925 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a22f98-5169-4d8d-8f58-ffada51947c3 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:47.934208 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a22f98-5169-4d8d-8f58-ffada51947c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:47.934484 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a22f98-5169-4d8d-8f58-ffada51947c3 None admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:47.935268 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a22f98-5169-4d8d-8f58-ffada51947c3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:47.935576 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a22f98-5169-4d8d-8f58-ffada51947c3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:47.936993 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a22f98-5169-4d8d-8f58-ffada51947c3 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:47.950312 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 437/871] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jul 15 19:15:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:47.954422 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 435/872] 10.222.0.33 () {60 vars in 1141 bytes} [Fri Jul 15 19:15:47 2022] POST /identity/v3/services => generated 251 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:15:47.956406 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a22f98-5169-4d8d-8f58-ffada51947c3 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:47.998082 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:48.085651 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a22f98-5169-4d8d-8f58-ffada51947c3 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:48.130437 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4920aae9-7e28-4a38-93ce-450ca2b7e7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:48.131014 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4920aae9-7e28-4a38-93ce-450ca2b7e7e2 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:48.131295 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4920aae9-7e28-4a38-93ce-450ca2b7e7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:48.131567 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4920aae9-7e28-4a38-93ce-450ca2b7e7e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:48.132399 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4920aae9-7e28-4a38-93ce-450ca2b7e7e2 None admin] RBAC: Adding query filter params (role.id=70de044d6962417593b6d03571e7502e, scope.project.id=55a756ccaef14cb49b26b0d68dd0b466, user.id=3d893c14620b4a129842062f14a68a78) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:48.132709 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4920aae9-7e28-4a38-93ce-450ca2b7e7e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:48.135638 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4920aae9-7e28-4a38-93ce-450ca2b7e7e2 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:48.142502 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 438/873] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 15 19:15:47 2022] GET /identity/v3/role_assignments?user.id=3d893c14620b4a129842062f14a68a78&scope.project.id=55a756ccaef14cb49b26b0d68dd0b466&role.id=70de044d6962417593b6d03571e7502e => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:15:48.736986 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22623445-a3f6-478b-a508-810352f45536 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:48.737519 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22623445-a3f6-478b-a508-810352f45536 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:48.737519 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22623445-a3f6-478b-a508-810352f45536 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:48.739349 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 436/874] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:15:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:15:48.745264 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51626b7-ee0c-414e-b246-f3c7587f0f31 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:48.745560 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51626b7-ee0c-414e-b246-f3c7587f0f31 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:48.745869 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51626b7-ee0c-414e-b246-f3c7587f0f31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:48.763446 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-f51626b7-ee0c-414e-b246-f3c7587f0f31 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:15:48.764630 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-f51626b7-ee0c-414e-b246-f3c7587f0f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:15:48.835916 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 439/875] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:15:48 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:15:48.858632 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22623445-a3f6-478b-a508-810352f45536 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:48.967533 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:48.975123 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22623445-a3f6-478b-a508-810352f45536 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:49.010780 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897fba81-2446-4229-ba16-4f0784370c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:49.011514 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897fba81-2446-4229-ba16-4f0784370c9d None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:49.011798 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897fba81-2446-4229-ba16-4f0784370c9d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:49.012072 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897fba81-2446-4229-ba16-4f0784370c9d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:49.012721 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897fba81-2446-4229-ba16-4f0784370c9d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:49.014124 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897fba81-2446-4229-ba16-4f0784370c9d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:49.027141 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-897fba81-2446-4229-ba16-4f0784370c9d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 19:15:49.027141 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:49.027141 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 19:15:49.027141 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 19:15:49.027141 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 19:15:49.027141 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 19:15:49.027141 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 19:15:49.027141 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:15:49.027141 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 19:15:49.027141 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:49.027141 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 19:15:49.027141 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:49.027141 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:49.027141 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:49.027141 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:49.027141 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:49.027141 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:49.028260 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:49.028260 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:49.028260 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:49.028260 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:49.028260 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:49.028260 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:49.028260 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 19:15:49.028260 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 19:15:49.028260 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 19:15:49.028260 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 19:15:49.028260 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:49.028260 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:49.028260 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:49.028260 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:49.028260 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:49.028260 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:49.029190 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:49.029190 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:49.029190 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:49.029190 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:49.029190 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:49.029190 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:49.029190 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:49.029190 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:49.029190 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:49.029190 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:49.029190 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 19:15:49.029190 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:15:49.029190 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 19:15:49.029190 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:49.029190 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 437/876] 10.222.0.33 () {56 vars in 1126 bytes} [Fri Jul 15 19:15:48 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:49.034624 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51626b7-ee0c-414e-b246-f3c7587f0f31 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:49.060059 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:49.139994 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51626b7-ee0c-414e-b246-f3c7587f0f31 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:49.176506 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da73cfa-64c7-43b2-8bed-b3083e2f769a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:49.177578 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da73cfa-64c7-43b2-8bed-b3083e2f769a None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:49.178100 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da73cfa-64c7-43b2-8bed-b3083e2f769a None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:49.178588 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da73cfa-64c7-43b2-8bed-b3083e2f769a None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:49.179507 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da73cfa-64c7-43b2-8bed-b3083e2f769a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:49.180020 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da73cfa-64c7-43b2-8bed-b3083e2f769a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:49.181606 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da73cfa-64c7-43b2-8bed-b3083e2f769a None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:49.187118 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 440/877] 10.222.0.33 () {56 vars in 1122 bytes} [Fri Jul 15 19:15:49 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:49.192694 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897fba81-2446-4229-ba16-4f0784370c9d None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:49.298204 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897fba81-2446-4229-ba16-4f0784370c9d None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:49.333376 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c66b19-67bc-4b2a-8492-ea49ebc9b721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:49.339199 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c66b19-67bc-4b2a-8492-ea49ebc9b721 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:49.339199 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c66b19-67bc-4b2a-8492-ea49ebc9b721 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:49.339199 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c66b19-67bc-4b2a-8492-ea49ebc9b721 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:49.339199 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c66b19-67bc-4b2a-8492-ea49ebc9b721 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:49.339199 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c66b19-67bc-4b2a-8492-ea49ebc9b721 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:49.339199 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c66b19-67bc-4b2a-8492-ea49ebc9b721 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:49.342574 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 438/878] 10.222.0.33 () {56 vars in 1122 bytes} [Fri Jul 15 19:15:49 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:49.638929 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be26753-924e-4254-b24d-970024c561cd None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:49.639326 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be26753-924e-4254-b24d-970024c561cd None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:49.639536 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be26753-924e-4254-b24d-970024c561cd None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:49.641136 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 441/879] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:15:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:15:49.647330 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075b481b-378f-469c-8a22-9143637f06cb None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:49.647619 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075b481b-378f-469c-8a22-9143637f06cb None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:49.647891 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075b481b-378f-469c-8a22-9143637f06cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:49.664561 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-075b481b-378f-469c-8a22-9143637f06cb None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:15:49.665746 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-075b481b-378f-469c-8a22-9143637f06cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:15:49.683903 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:49.734664 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 439/880] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:15:49 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:15:49.756819 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be26753-924e-4254-b24d-970024c561cd None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:49.846400 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:49.862866 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be26753-924e-4254-b24d-970024c561cd None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:49.898810 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75efe8a7-02a5-41ae-a73c-f5e3164df72d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:49.899403 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75efe8a7-02a5-41ae-a73c-f5e3164df72d None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:49.899685 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75efe8a7-02a5-41ae-a73c-f5e3164df72d None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:49.899957 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75efe8a7-02a5-41ae-a73c-f5e3164df72d None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:49.900697 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75efe8a7-02a5-41ae-a73c-f5e3164df72d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:49.902159 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75efe8a7-02a5-41ae-a73c-f5e3164df72d None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:50.003977 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 442/881] 10.222.0.33 () {60 vars in 1141 bytes} [Fri Jul 15 19:15:49 2022] POST /identity/v3/services => generated 269 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:15:50.065935 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846a044a-30f4-472d-84b4-a92b038ba127 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:50.066988 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846a044a-30f4-472d-84b4-a92b038ba127 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:50.067494 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846a044a-30f4-472d-84b4-a92b038ba127 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:50.069585 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 440/882] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:15:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:15:50.076572 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a69be53-d50d-4389-a524-082a7ea78da0 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:50.078097 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a69be53-d50d-4389-a524-082a7ea78da0 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:50.078873 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a69be53-d50d-4389-a524-082a7ea78da0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:50.100916 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-8a69be53-d50d-4389-a524-082a7ea78da0 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:15:50.102546 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-8a69be53-d50d-4389-a524-082a7ea78da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:15:50.183261 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 443/883] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:15:50 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:15:50.205523 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846a044a-30f4-472d-84b4-a92b038ba127 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:50.316190 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846a044a-30f4-472d-84b4-a92b038ba127 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:50.358117 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941b68cc-ad14-4762-a812-f37b947a261b None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:50.358471 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941b68cc-ad14-4762-a812-f37b947a261b None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:50.358678 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941b68cc-ad14-4762-a812-f37b947a261b None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:50.359465 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc570aa0-4b60-44a3-baa7-1d64344e8a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:50.360292 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 444/884] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:15:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 19:15:50.360792 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc570aa0-4b60-44a3-baa7-1d64344e8a0b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:50.361321 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc570aa0-4b60-44a3-baa7-1d64344e8a0b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:50.361895 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc570aa0-4b60-44a3-baa7-1d64344e8a0b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:50.362753 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc570aa0-4b60-44a3-baa7-1d64344e8a0b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:50.364406 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc570aa0-4b60-44a3-baa7-1d64344e8a0b None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:50.365275 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d665b0d6-7c49-4bce-8c47-55112515d7d2 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:50.365550 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d665b0d6-7c49-4bce-8c47-55112515d7d2 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:50.365861 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d665b0d6-7c49-4bce-8c47-55112515d7d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:50.380722 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-bc570aa0-4b60-44a3-baa7-1d64344e8a0b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 15 19:15:50.380722 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:50.380722 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 19:15:50.380722 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 19:15:50.380722 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 19:15:50.380722 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 19:15:50.380722 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 19:15:50.380722 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:15:50.380722 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 15 19:15:50.380722 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:50.380722 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 19:15:50.380722 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:50.380722 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:50.380722 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:50.380722 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:50.380722 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:50.380722 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:50.381982 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:50.381982 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:50.381982 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:50.381982 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:50.381982 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:50.381982 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:50.381982 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 19:15:50.381982 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 19:15:50.381982 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 19:15:50.381982 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 19:15:50.381982 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:50.381982 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:50.381982 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:50.381982 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:50.381982 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:50.381982 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:50.382827 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:50.382827 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:50.382827 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:50.382827 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:50.382827 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:50.382827 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:50.382827 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:50.382827 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:50.382827 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:50.382827 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:50.382827 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 19:15:50.382827 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:15:50.382827 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 15 19:15:50.382827 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:50.385128 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-d665b0d6-7c49-4bce-8c47-55112515d7d2 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:15:50.385512 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 441/885] 10.222.0.33 () {56 vars in 1114 bytes} [Fri Jul 15 19:15:50 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:50.389588 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-d665b0d6-7c49-4bce-8c47-55112515d7d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:15:50.392729 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc570aa0-4b60-44a3-baa7-1d64344e8a0b None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:50.414888 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:50.484826 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 445/886] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:15:50 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 19:15:50.508764 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d665b0d6-7c49-4bce-8c47-55112515d7d2 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:50.511208 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc570aa0-4b60-44a3-baa7-1d64344e8a0b None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:50.550265 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:50.559531 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b90c06-b37c-4ade-a59c-b855ded722ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:50.560537 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b90c06-b37c-4ade-a59c-b855ded722ae None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:50.560872 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b90c06-b37c-4ade-a59c-b855ded722ae None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:50.561156 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b90c06-b37c-4ade-a59c-b855ded722ae None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:50.561905 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b90c06-b37c-4ade-a59c-b855ded722ae None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:50.562218 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b90c06-b37c-4ade-a59c-b855ded722ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:50.565440 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b90c06-b37c-4ade-a59c-b855ded722ae None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:50.571224 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 442/887] 10.222.0.33 () {56 vars in 1114 bytes} [Fri Jul 15 19:15:50 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:50.578508 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b90c06-b37c-4ade-a59c-b855ded722ae None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:50.630885 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d665b0d6-7c49-4bce-8c47-55112515d7d2 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:50.667558 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ea8075-0ecc-4f4f-874a-6665de3464a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:50.668715 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ea8075-0ecc-4f4f-874a-6665de3464a5 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:50.669148 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ea8075-0ecc-4f4f-874a-6665de3464a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:50.669612 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ea8075-0ecc-4f4f-874a-6665de3464a5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:50.669986 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:50.682056 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ea8075-0ecc-4f4f-874a-6665de3464a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:15:50.688820 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b90c06-b37c-4ade-a59c-b855ded722ae None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:50.690969 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-50ea8075-0ecc-4f4f-874a-6665de3464a5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 15 19:15:50.690969 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:50.690969 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:50.690969 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:50.690969 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:50.690969 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:50.690969 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:50.690969 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:50.690969 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:50.690969 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:50.690969 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:50.690969 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:50.690969 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:15:50.690969 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:15:50.690969 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:15:50.690969 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:15:50.692738 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:15:50.692738 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:15:50.692738 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:15:50.692738 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:15:50.692738 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:50.692738 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:50.692738 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:50.692738 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:50.692738 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:50.692738 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:50.692738 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:50.692738 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:50.692738 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:50.692738 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:50.692738 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:50.692738 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:50.693623 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:50.693623 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:50.693623 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:50.693623 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:50.693623 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:15:50.693623 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:15:50.693623 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:15:50.693623 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:15:50.693623 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:15:50.693623 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:15:50.693623 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 15 19:15:50.693623 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:50.695921 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 446/888] 10.222.0.33 () {56 vars in 1120 bytes} [Fri Jul 15 19:15:50 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:15:50.702021 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ea8075-0ecc-4f4f-874a-6665de3464a5 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:50.732031 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebe2b60-c593-4d9b-aa53-857b6241f696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:50.733655 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebe2b60-c593-4d9b-aa53-857b6241f696 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:50.734492 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebe2b60-c593-4d9b-aa53-857b6241f696 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:50.735094 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebe2b60-c593-4d9b-aa53-857b6241f696 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:50.736598 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebe2b60-c593-4d9b-aa53-857b6241f696 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:50.737335 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebe2b60-c593-4d9b-aa53-857b6241f696 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:50.739464 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebe2b60-c593-4d9b-aa53-857b6241f696 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:50.746918 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 443/889] 10.222.0.33 () {56 vars in 1114 bytes} [Fri Jul 15 19:15:50 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:15:50.752305 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebe2b60-c593-4d9b-aa53-857b6241f696 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:50.826256 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ea8075-0ecc-4f4f-874a-6665de3464a5 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:50.830152 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:50.863063 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ac7292-9939-4c2a-a67b-9f4873db3fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:50.863945 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ac7292-9939-4c2a-a67b-9f4873db3fbf None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:50.864321 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ac7292-9939-4c2a-a67b-9f4873db3fbf None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:50.864693 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ac7292-9939-4c2a-a67b-9f4873db3fbf None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:50.865694 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ac7292-9939-4c2a-a67b-9f4873db3fbf None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:50.866317 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ac7292-9939-4c2a-a67b-9f4873db3fbf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:50.867828 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ac7292-9939-4c2a-a67b-9f4873db3fbf None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:50.879415 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebe2b60-c593-4d9b-aa53-857b6241f696 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:50.882621 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 447/890] 10.222.0.33 () {56 vars in 1115 bytes} [Fri Jul 15 19:15:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:50.883866 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ac7292-9939-4c2a-a67b-9f4873db3fbf None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:50.938332 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06576636-3808-4eb8-9ebb-578c57ef19c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:50.939632 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06576636-3808-4eb8-9ebb-578c57ef19c7 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:50.940086 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06576636-3808-4eb8-9ebb-578c57ef19c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:50.940511 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06576636-3808-4eb8-9ebb-578c57ef19c7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:50.941517 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06576636-3808-4eb8-9ebb-578c57ef19c7 None admin] RBAC: Adding query filter params (interface=public, service_id=3e55de2b96584bedbc9017d2cbadc47b, region_id=RegionOne) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:50.942004 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06576636-3808-4eb8-9ebb-578c57ef19c7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:50.945023 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06576636-3808-4eb8-9ebb-578c57ef19c7 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:50.957866 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 444/891] 10.222.0.33 () {56 vars in 1251 bytes} [Fri Jul 15 19:15:50 2022] GET /identity/v3/endpoints?service_id=3e55de2b96584bedbc9017d2cbadc47b&interface=public®ion_id=RegionOne => generated 194 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:50.960317 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06576636-3808-4eb8-9ebb-578c57ef19c7 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:50.977817 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:50.998947 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ac7292-9939-4c2a-a67b-9f4873db3fbf None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:51.026235 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:51.038463 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab741ac-30a5-446c-b7a5-9017b008da40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:51.039448 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab741ac-30a5-446c-b7a5-9017b008da40 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:51.039731 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab741ac-30a5-446c-b7a5-9017b008da40 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:51.040196 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab741ac-30a5-446c-b7a5-9017b008da40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:51.051203 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab741ac-30a5-446c-b7a5-9017b008da40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:51.052915 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab741ac-30a5-446c-b7a5-9017b008da40 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:51.073943 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-0ab741ac-30a5-446c-b7a5-9017b008da40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:51.073943 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:51.073943 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:51.073943 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:51.073943 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:51.073943 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:51.073943 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:51.073943 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:51.073943 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:51.073943 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:51.073943 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:51.073943 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:51.073943 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:15:51.073943 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:15:51.073943 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:15:51.074920 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:15:51.074920 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:51.074920 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:51.074920 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:51.074920 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:51.074920 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:51.074920 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:51.074920 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:51.074920 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:51.074920 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:51.074920 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:51.074920 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:51.074920 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:51.074920 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:51.074920 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:51.074920 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:51.074920 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:51.075803 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:15:51.075803 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:15:51.075803 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:51.075803 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:51.080343 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 448/892] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:15:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:15:51.080907 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab741ac-30a5-446c-b7a5-9017b008da40 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:51.095988 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06576636-3808-4eb8-9ebb-578c57ef19c7 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:51.141277 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9111b465-6011-443f-8086-c60dbec39ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:51.143232 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9111b465-6011-443f-8086-c60dbec39ff6 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:51.143511 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9111b465-6011-443f-8086-c60dbec39ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:51.144149 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9111b465-6011-443f-8086-c60dbec39ff6 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:51.145111 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9111b465-6011-443f-8086-c60dbec39ff6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:51.146812 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9111b465-6011-443f-8086-c60dbec39ff6 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:51.158345 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 445/893] 10.222.0.33 () {56 vars in 1087 bytes} [Fri Jul 15 19:15:50 2022] GET /identity/v3/services => generated 1268 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:15:51.197303 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab741ac-30a5-446c-b7a5-9017b008da40 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:51.243864 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46212394-9ee1-45af-b3ba-cbb2dc8d23ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:51.244966 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46212394-9ee1-45af-b3ba-cbb2dc8d23ff None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:51.245453 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46212394-9ee1-45af-b3ba-cbb2dc8d23ff None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:51.245991 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46212394-9ee1-45af-b3ba-cbb2dc8d23ff None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:51.246979 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46212394-9ee1-45af-b3ba-cbb2dc8d23ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:51.247518 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46212394-9ee1-45af-b3ba-cbb2dc8d23ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:51.249172 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46212394-9ee1-45af-b3ba-cbb2dc8d23ff None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:51.265397 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 449/894] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:15:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:51.277905 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9111b465-6011-443f-8086-c60dbec39ff6 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:51.360396 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:51.386100 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9111b465-6011-443f-8086-c60dbec39ff6 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:51.421068 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2796b385-519b-4176-9887-2395b3c68705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:51.421909 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2796b385-519b-4176-9887-2395b3c68705 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:51.422187 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2796b385-519b-4176-9887-2395b3c68705 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:51.422457 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2796b385-519b-4176-9887-2395b3c68705 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:51.430793 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2796b385-519b-4176-9887-2395b3c68705 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:51.432214 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2796b385-519b-4176-9887-2395b3c68705 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:51.447331 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-2796b385-519b-4176-9887-2395b3c68705 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 15 19:15:51.447331 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:51.447331 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:51.447331 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:51.447331 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:51.447331 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:51.447331 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:51.447331 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:51.447331 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:51.447331 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:51.447331 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:51.447331 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:51.447331 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 19:15:51.447331 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 19:15:51.447331 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 19:15:51.447331 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 19:15:51.448326 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:51.448326 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:51.448326 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:15:51.448326 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:15:51.448326 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:15:51.448326 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:15:51.448326 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:51.448326 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:51.448326 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:51.448326 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:51.448326 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:51.448326 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:51.448326 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:51.448326 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:51.448326 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:51.448326 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:51.448326 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:51.448326 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:51.449968 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:51.449968 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:51.449968 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 19:15:51.449968 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 19:15:51.449968 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 19:15:51.449968 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 19:15:51.449968 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 19:15:51.449968 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 19:15:51.449968 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 15 19:15:51.449968 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:51.449968 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 446/895] 10.222.0.33 () {56 vars in 1117 bytes} [Fri Jul 15 19:15:51 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:15:51.454482 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46212394-9ee1-45af-b3ba-cbb2dc8d23ff None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:51.465541 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba48100f-b1c1-478a-9908-a401946573ed None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:51.465859 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba48100f-b1c1-478a-9908-a401946573ed None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:51.466142 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba48100f-b1c1-478a-9908-a401946573ed None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:51.467751 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 447/896] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:15:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 19:15:51.472815 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b507114-768f-4451-91e3-58476b2cb723 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:51.473107 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b507114-768f-4451-91e3-58476b2cb723 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:51.473386 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b507114-768f-4451-91e3-58476b2cb723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:51.490995 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-6b507114-768f-4451-91e3-58476b2cb723 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:15:51.492368 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-6b507114-768f-4451-91e3-58476b2cb723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:15:51.502196 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:51.574016 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 448/897] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:15:51 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 19:15:51.580697 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46212394-9ee1-45af-b3ba-cbb2dc8d23ff None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:51.590758 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b507114-768f-4451-91e3-58476b2cb723 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:51.623367 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbd7096-3222-4637-9278-844a5faed18d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:51.624319 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbd7096-3222-4637-9278-844a5faed18d None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:51.624668 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbd7096-3222-4637-9278-844a5faed18d None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:51.625716 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbd7096-3222-4637-9278-844a5faed18d None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:51.626599 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbd7096-3222-4637-9278-844a5faed18d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:51.626905 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbd7096-3222-4637-9278-844a5faed18d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:51.628530 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbd7096-3222-4637-9278-844a5faed18d None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:51.650763 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 450/898] 10.222.0.33 () {56 vars in 1149 bytes} [Fri Jul 15 19:15:51 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:51.657333 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbd7096-3222-4637-9278-844a5faed18d None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:51.667496 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:51.708986 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b507114-768f-4451-91e3-58476b2cb723 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:51.745961 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d36e25-f3fc-4bc0-81e9-80840e3dbd57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:51.746900 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d36e25-f3fc-4bc0-81e9-80840e3dbd57 None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:51.747354 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d36e25-f3fc-4bc0-81e9-80840e3dbd57 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:51.747791 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d36e25-f3fc-4bc0-81e9-80840e3dbd57 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:51.748754 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d36e25-f3fc-4bc0-81e9-80840e3dbd57 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:51.750332 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d36e25-f3fc-4bc0-81e9-80840e3dbd57 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:51.768048 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:51.768048 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbd7096-3222-4637-9278-844a5faed18d None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:51.811224 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74ead52-b720-416c-b2af-c39103b2cad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:51.811884 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74ead52-b720-416c-b2af-c39103b2cad2 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:51.812168 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74ead52-b720-416c-b2af-c39103b2cad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:51.812620 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74ead52-b720-416c-b2af-c39103b2cad2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:51.827819 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 449/899] 10.222.0.33 () {60 vars in 1141 bytes} [Fri Jul 15 19:15:51 2022] POST /identity/v3/services => generated 241 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:15:51.897928 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74ead52-b720-416c-b2af-c39103b2cad2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:51.899404 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74ead52-b720-416c-b2af-c39103b2cad2 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:52.003423 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:52.012757 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 451/900] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 15 19:15:51 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 19:15:52.018702 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d36e25-f3fc-4bc0-81e9-80840e3dbd57 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:52.143707 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d36e25-f3fc-4bc0-81e9-80840e3dbd57 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:52.168865 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:52.171077 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e459020-3d69-4613-95ce-8140005033e3 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:52.172134 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e459020-3d69-4613-95ce-8140005033e3 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:52.172655 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e459020-3d69-4613-95ce-8140005033e3 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:52.175432 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 452/901] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:15:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 19:15:52.181576 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b66860-7abe-4571-81ba-b89bf8c19204 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:52.182012 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b66860-7abe-4571-81ba-b89bf8c19204 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:52.182390 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b66860-7abe-4571-81ba-b89bf8c19204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:52.184099 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06371e22-1490-4ca8-90fb-d1a87ca21b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:52.185069 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06371e22-1490-4ca8-90fb-d1a87ca21b6d None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:52.185503 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06371e22-1490-4ca8-90fb-d1a87ca21b6d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:52.185973 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06371e22-1490-4ca8-90fb-d1a87ca21b6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:52.198018 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06371e22-1490-4ca8-90fb-d1a87ca21b6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:52.199708 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06371e22-1490-4ca8-90fb-d1a87ca21b6d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:52.201041 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-28b66860-7abe-4571-81ba-b89bf8c19204 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:15:52.202497 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-28b66860-7abe-4571-81ba-b89bf8c19204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:15:52.220149 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-06371e22-1490-4ca8-90fb-d1a87ca21b6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:52.220149 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:52.220149 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:52.220149 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:52.220149 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:52.220149 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:52.220149 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:52.220149 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:52.220149 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:52.220149 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:52.220149 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:52.220149 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:52.220149 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:15:52.220149 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:15:52.220149 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:15:52.221124 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:15:52.221124 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:52.221124 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:52.221124 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:52.221124 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:52.221124 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:52.221124 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:52.221124 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:52.221124 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:52.221124 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:52.221124 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:52.221124 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:52.221124 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:52.221124 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:52.221124 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:52.221124 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:52.221124 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:52.222075 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:15:52.222075 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:15:52.222075 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:52.222075 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:52.222075 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 450/902] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:15:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:52.226887 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06371e22-1490-4ca8-90fb-d1a87ca21b6d None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:52.286035 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 453/903] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:15:52 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 19:15:52.302139 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b66860-7abe-4571-81ba-b89bf8c19204 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:52.340568 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06371e22-1490-4ca8-90fb-d1a87ca21b6d None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:52.381613 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fc744c-e4f9-4557-965b-a3be5e918260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:52.382358 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc744c-e4f9-4557-965b-a3be5e918260 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:52.382635 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc744c-e4f9-4557-965b-a3be5e918260 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:52.382911 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc744c-e4f9-4557-965b-a3be5e918260 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:52.383710 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fc744c-e4f9-4557-965b-a3be5e918260 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:52.384179 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fc744c-e4f9-4557-965b-a3be5e918260 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:52.385755 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fc744c-e4f9-4557-965b-a3be5e918260 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:52.405237 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 451/904] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:15:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:52.412584 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b66860-7abe-4571-81ba-b89bf8c19204 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:52.412923 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fc744c-e4f9-4557-965b-a3be5e918260 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:52.441131 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:52.453203 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:52.454253 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97608a2-876a-40e5-8b06-0d14f718306d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:52.455411 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97608a2-876a-40e5-8b06-0d14f718306d None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:52.455697 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97608a2-876a-40e5-8b06-0d14f718306d None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:52.455976 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97608a2-876a-40e5-8b06-0d14f718306d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:52.456697 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97608a2-876a-40e5-8b06-0d14f718306d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:52.458367 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97608a2-876a-40e5-8b06-0d14f718306d None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:52.478754 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-a97608a2-876a-40e5-8b06-0d14f718306d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 15 19:15:52.478754 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:52.478754 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 19:15:52.478754 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 19:15:52.478754 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 19:15:52.478754 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 19:15:52.478754 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 19:15:52.478754 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:15:52.478754 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 15 19:15:52.478754 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:52.478754 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 19:15:52.478754 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:52.478754 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:52.478754 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:52.478754 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:52.478754 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:52.480606 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:52.480606 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:52.480606 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:52.480606 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:52.480606 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:52.480606 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:52.480606 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:52.480606 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 19:15:52.480606 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 19:15:52.480606 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 19:15:52.480606 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 19:15:52.480606 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:52.480606 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:52.480606 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:52.480606 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:52.480606 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:52.481500 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:52.481500 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:52.481500 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:52.481500 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:52.481500 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:52.481500 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:52.481500 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:52.481500 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:52.481500 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:52.481500 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:52.481500 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:52.481500 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 19:15:52.481500 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:15:52.481500 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 15 19:15:52.481500 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:52.484426 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 454/905] 10.222.0.33 () {56 vars in 1132 bytes} [Fri Jul 15 19:15:52 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:15:52.490508 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97608a2-876a-40e5-8b06-0d14f718306d None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:52.545196 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fc744c-e4f9-4557-965b-a3be5e918260 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:52.591203 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98738363-9f81-45fe-ba79-df548ec3554f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:52.592482 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98738363-9f81-45fe-ba79-df548ec3554f None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:52.592982 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98738363-9f81-45fe-ba79-df548ec3554f None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:52.593474 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98738363-9f81-45fe-ba79-df548ec3554f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:52.602498 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98738363-9f81-45fe-ba79-df548ec3554f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:52.604302 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98738363-9f81-45fe-ba79-df548ec3554f None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:52.607870 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97608a2-876a-40e5-8b06-0d14f718306d None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:52.620199 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-98738363-9f81-45fe-ba79-df548ec3554f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:15:52.620199 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:52.620199 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:52.620199 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:52.620199 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:52.620199 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:52.620199 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:52.620199 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:52.620199 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:52.620199 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:52.620199 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:52.620199 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:52.620199 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 19:15:52.620199 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 19:15:52.620199 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 19:15:52.621226 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 19:15:52.621226 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:52.621226 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:52.621226 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:52.621226 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:52.621226 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:52.621226 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:52.621226 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:52.621226 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:52.621226 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:52.621226 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:52.621226 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:52.621226 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:52.621226 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:52.621226 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:52.621226 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:52.621226 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:52.622202 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 19:15:52.622202 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 19:15:52.622202 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:15:52.622202 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:15:52.622202 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:15:52.622202 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:15:52.622202 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:15:52.622202 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:52.625006 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 452/906] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 15 19:15:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:52.631052 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98738363-9f81-45fe-ba79-df548ec3554f None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:52.655960 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c801eb-d41c-4383-8cc8-e9e667534565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:52.657449 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c801eb-d41c-4383-8cc8-e9e667534565 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:52.658007 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c801eb-d41c-4383-8cc8-e9e667534565 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:52.658508 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c801eb-d41c-4383-8cc8-e9e667534565 None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:52.660588 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c801eb-d41c-4383-8cc8-e9e667534565 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:52.661304 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c801eb-d41c-4383-8cc8-e9e667534565 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:52.663368 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c801eb-d41c-4383-8cc8-e9e667534565 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:52.676569 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 455/907] 10.222.0.33 () {56 vars in 1126 bytes} [Fri Jul 15 19:15:52 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:15:52.678249 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c801eb-d41c-4383-8cc8-e9e667534565 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:52.740506 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:52.755297 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:52.767027 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98738363-9f81-45fe-ba79-df548ec3554f None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:52.799873 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c801eb-d41c-4383-8cc8-e9e667534565 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:52.812586 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6964bf1-63e9-4cf7-9f79-3e91313665d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:52.813282 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6964bf1-63e9-4cf7-9f79-3e91313665d7 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:52.813666 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6964bf1-63e9-4cf7-9f79-3e91313665d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:52.814105 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6964bf1-63e9-4cf7-9f79-3e91313665d7 None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:52.815104 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6964bf1-63e9-4cf7-9f79-3e91313665d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:52.815636 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6964bf1-63e9-4cf7-9f79-3e91313665d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:52.817198 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6964bf1-63e9-4cf7-9f79-3e91313665d7 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:52.836802 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 453/908] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jul 15 19:15:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:52.838068 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6964bf1-63e9-4cf7-9f79-3e91313665d7 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:52.843754 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e998a4-d672-43e5-81a2-60f1473512b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:52.844630 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e998a4-d672-43e5-81a2-60f1473512b1 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:52.845088 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e998a4-d672-43e5-81a2-60f1473512b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:52.845513 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e998a4-d672-43e5-81a2-60f1473512b1 None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:52.846422 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e998a4-d672-43e5-81a2-60f1473512b1 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:52.846907 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e998a4-d672-43e5-81a2-60f1473512b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:52.849018 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e998a4-d672-43e5-81a2-60f1473512b1 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:52.861226 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 456/909] 10.222.0.33 () {56 vars in 1126 bytes} [Fri Jul 15 19:15:52 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:52.862687 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e998a4-d672-43e5-81a2-60f1473512b1 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:52.966679 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6964bf1-63e9-4cf7-9f79-3e91313665d7 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:52.999055 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e998a4-d672-43e5-81a2-60f1473512b1 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:53.007051 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c433bbb-263d-4edc-904a-7267e005d227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:53.007725 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c433bbb-263d-4edc-904a-7267e005d227 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:53.008009 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c433bbb-263d-4edc-904a-7267e005d227 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:53.008284 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c433bbb-263d-4edc-904a-7267e005d227 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:53.009127 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c433bbb-263d-4edc-904a-7267e005d227 None admin] RBAC: Adding query filter params (role.id=c1dc7198be2d43d3aa44d16107d9a959, scope.project.id=55a756ccaef14cb49b26b0d68dd0b466, user.id=3d893c14620b4a129842062f14a68a78) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:53.009433 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c433bbb-263d-4edc-904a-7267e005d227 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:53.013010 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c433bbb-263d-4edc-904a-7267e005d227 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:53.018094 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 454/910] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 15 19:15:52 2022] GET /identity/v3/role_assignments?user.id=3d893c14620b4a129842062f14a68a78&scope.project.id=55a756ccaef14cb49b26b0d68dd0b466&role.id=c1dc7198be2d43d3aa44d16107d9a959 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:53.019374 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:53.043624 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c05a79b-31ed-4e16-9319-bc98380729a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:53.044593 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c05a79b-31ed-4e16-9319-bc98380729a4 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:53.045063 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c05a79b-31ed-4e16-9319-bc98380729a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:53.045507 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c05a79b-31ed-4e16-9319-bc98380729a4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:53.046539 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c05a79b-31ed-4e16-9319-bc98380729a4 None admin] RBAC: Adding query filter params (interface=public, service_id=a5aff149885840a1b8d7a0e43e346143, region_id=RegionOne) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:53.047015 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c05a79b-31ed-4e16-9319-bc98380729a4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:53.048571 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c05a79b-31ed-4e16-9319-bc98380729a4 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:53.059321 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 457/911] 10.222.0.33 () {56 vars in 1251 bytes} [Fri Jul 15 19:15:52 2022] GET /identity/v3/endpoints?service_id=a5aff149885840a1b8d7a0e43e346143&interface=public®ion_id=RegionOne => generated 194 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:53.065421 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c433bbb-263d-4edc-904a-7267e005d227 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:53.088557 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:53.178961 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c433bbb-263d-4edc-904a-7267e005d227 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:53.218435 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd12dea-f88d-4f78-9b9a-bcc031436aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:53.219114 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd12dea-f88d-4f78-9b9a-bcc031436aed None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:53.219398 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd12dea-f88d-4f78-9b9a-bcc031436aed None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:53.219673 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd12dea-f88d-4f78-9b9a-bcc031436aed None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:53.220357 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd12dea-f88d-4f78-9b9a-bcc031436aed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:53.221729 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd12dea-f88d-4f78-9b9a-bcc031436aed None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:53.228875 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 455/912] 10.222.0.33 () {56 vars in 1087 bytes} [Fri Jul 15 19:15:53 2022] GET /identity/v3/services => generated 1497 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:15:53.301817 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392aef0c-2749-4a2f-9916-62b06c8fc727 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:53.302329 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392aef0c-2749-4a2f-9916-62b06c8fc727 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:53.302329 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392aef0c-2749-4a2f-9916-62b06c8fc727 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:53.303960 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 458/913] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:15:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:15:53.309695 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fe64e9-cb63-455e-94ce-ee62658462a7 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:53.310269 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fe64e9-cb63-455e-94ce-ee62658462a7 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:53.310733 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fe64e9-cb63-455e-94ce-ee62658462a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:53.330207 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-58fe64e9-cb63-455e-94ce-ee62658462a7 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:15:53.331681 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-58fe64e9-cb63-455e-94ce-ee62658462a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:15:53.407639 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 456/914] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:15:53 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:15:53.430794 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392aef0c-2749-4a2f-9916-62b06c8fc727 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:53.546852 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392aef0c-2749-4a2f-9916-62b06c8fc727 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:53.583655 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42878e0-0440-4ac7-858d-001ec5a874b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:53.584442 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42878e0-0440-4ac7-858d-001ec5a874b3 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:53.584725 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42878e0-0440-4ac7-858d-001ec5a874b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:53.585050 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42878e0-0440-4ac7-858d-001ec5a874b3 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:53.585723 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42878e0-0440-4ac7-858d-001ec5a874b3 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:53.587239 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42878e0-0440-4ac7-858d-001ec5a874b3 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:53.599874 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-b42878e0-0440-4ac7-858d-001ec5a874b3 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 15 19:15:53.599874 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:53.599874 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 19:15:53.599874 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 19:15:53.599874 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 19:15:53.599874 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 19:15:53.599874 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 19:15:53.599874 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:15:53.599874 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 15 19:15:53.599874 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:53.599874 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 19:15:53.599874 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:53.599874 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:53.599874 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:53.599874 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:53.599874 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:53.599874 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:53.600906 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:53.600906 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:53.600906 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:53.600906 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:53.600906 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:53.600906 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:53.600906 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 19:15:53.600906 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 19:15:53.600906 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 19:15:53.600906 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 19:15:53.600906 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:53.600906 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:53.600906 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:53.600906 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:53.600906 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:53.600906 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:53.601746 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:53.601746 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:53.601746 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:53.601746 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:53.601746 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:53.601746 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:53.601746 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:53.601746 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:53.601746 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:53.601746 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:53.601746 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 19:15:53.601746 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:15:53.601746 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 15 19:15:53.601746 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:53.601746 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 459/915] 10.222.0.33 () {56 vars in 1114 bytes} [Fri Jul 15 19:15:53 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:53.609113 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fe64e9-cb63-455e-94ce-ee62658462a7 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:53.626164 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:53.719618 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fe64e9-cb63-455e-94ce-ee62658462a7 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:53.757104 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf4e1ce-adfa-4cc6-aa4b-72170784b337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:53.757859 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf4e1ce-adfa-4cc6-aa4b-72170784b337 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:53.758144 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf4e1ce-adfa-4cc6-aa4b-72170784b337 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:53.758419 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf4e1ce-adfa-4cc6-aa4b-72170784b337 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:53.759186 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf4e1ce-adfa-4cc6-aa4b-72170784b337 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:53.759494 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf4e1ce-adfa-4cc6-aa4b-72170784b337 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:53.760933 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf4e1ce-adfa-4cc6-aa4b-72170784b337 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:53.766430 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 457/916] 10.222.0.33 () {56 vars in 1114 bytes} [Fri Jul 15 19:15:53 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:53.776841 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42878e0-0440-4ac7-858d-001ec5a874b3 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:53.829332 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:53.886009 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42878e0-0440-4ac7-858d-001ec5a874b3 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:53.921353 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1469e00d-20a4-4957-baab-bdafaaf4ba13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:53.922409 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1469e00d-20a4-4957-baab-bdafaaf4ba13 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:53.922956 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1469e00d-20a4-4957-baab-bdafaaf4ba13 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:53.923428 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1469e00d-20a4-4957-baab-bdafaaf4ba13 None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:53.924345 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1469e00d-20a4-4957-baab-bdafaaf4ba13 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:53.924945 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1469e00d-20a4-4957-baab-bdafaaf4ba13 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:53.927082 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1469e00d-20a4-4957-baab-bdafaaf4ba13 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:53.932546 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 460/917] 10.222.0.33 () {56 vars in 1114 bytes} [Fri Jul 15 19:15:53 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:53.938417 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf4e1ce-adfa-4cc6-aa4b-72170784b337 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:54.016202 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:54.019336 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add24589-36c7-42bb-89f8-e28753e6713d None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:54.019630 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add24589-36c7-42bb-89f8-e28753e6713d None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:54.019911 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add24589-36c7-42bb-89f8-e28753e6713d None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:54.025327 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 461/918] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:15:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 19:15:54.031604 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20974cf0-4cc8-4526-9269-5cbf15ef6b76 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:54.031604 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20974cf0-4cc8-4526-9269-5cbf15ef6b76 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:54.031604 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20974cf0-4cc8-4526-9269-5cbf15ef6b76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:54.046419 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-20974cf0-4cc8-4526-9269-5cbf15ef6b76 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:15:54.048565 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-20974cf0-4cc8-4526-9269-5cbf15ef6b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:15:54.054097 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf4e1ce-adfa-4cc6-aa4b-72170784b337 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:54.100130 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb90b3bf-0620-4533-a23e-d2ac8b1a156f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:54.101160 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb90b3bf-0620-4533-a23e-d2ac8b1a156f None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:54.102149 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb90b3bf-0620-4533-a23e-d2ac8b1a156f None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:54.102606 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb90b3bf-0620-4533-a23e-d2ac8b1a156f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:54.103997 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb90b3bf-0620-4533-a23e-d2ac8b1a156f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:54.106097 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb90b3bf-0620-4533-a23e-d2ac8b1a156f None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:54.137587 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 462/919] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:15:54 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 19:15:54.160735 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20974cf0-4cc8-4526-9269-5cbf15ef6b76 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:54.200294 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 458/920] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 15 19:15:53 2022] POST /identity/v3/endpoints => generated 346 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:15:54.208856 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:54.273209 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20974cf0-4cc8-4526-9269-5cbf15ef6b76 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:54.317611 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def9681c-2aa8-416a-88d7-2317b4343ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:54.318462 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def9681c-2aa8-416a-88d7-2317b4343ee9 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:54.318749 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def9681c-2aa8-416a-88d7-2317b4343ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:54.319026 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def9681c-2aa8-416a-88d7-2317b4343ee9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:54.319742 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def9681c-2aa8-416a-88d7-2317b4343ee9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:54.321173 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def9681c-2aa8-416a-88d7-2317b4343ee9 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:54.334693 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-def9681c-2aa8-416a-88d7-2317b4343ee9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 19:15:54.334693 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:54.334693 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 19:15:54.334693 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 19:15:54.334693 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 19:15:54.334693 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 19:15:54.334693 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 19:15:54.334693 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:15:54.334693 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 19:15:54.334693 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:54.334693 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 19:15:54.334693 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:54.334693 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:54.334693 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:54.334693 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:54.334693 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:54.334693 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:54.335758 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:54.335758 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:54.335758 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:54.335758 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:54.335758 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:54.335758 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:54.335758 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 19:15:54.335758 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 19:15:54.335758 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 19:15:54.335758 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 19:15:54.335758 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:54.335758 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:54.335758 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:54.335758 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:54.335758 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:54.335758 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:54.336588 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:54.336588 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:54.336588 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:54.336588 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:54.336588 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:54.336588 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:54.336588 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:54.336588 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:54.336588 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:54.336588 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:54.336588 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 19:15:54.336588 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:15:54.336588 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 19:15:54.336588 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:54.339113 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 463/921] 10.222.0.33 () {56 vars in 1126 bytes} [Fri Jul 15 19:15:54 2022] GET /identity/v3/services/object-store => generated 93 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:15:54.344809 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb90b3bf-0620-4533-a23e-d2ac8b1a156f None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:54.486009 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb90b3bf-0620-4533-a23e-d2ac8b1a156f None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:54.516215 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:54.522920 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661e5c86-1b74-4872-a32a-06ea519c6668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:54.523639 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661e5c86-1b74-4872-a32a-06ea519c6668 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:54.523928 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661e5c86-1b74-4872-a32a-06ea519c6668 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:54.524208 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661e5c86-1b74-4872-a32a-06ea519c6668 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:54.524923 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661e5c86-1b74-4872-a32a-06ea519c6668 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:54.525228 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661e5c86-1b74-4872-a32a-06ea519c6668 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:54.526718 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661e5c86-1b74-4872-a32a-06ea519c6668 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:54.531819 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 459/922] 10.222.0.33 () {56 vars in 1122 bytes} [Fri Jul 15 19:15:54 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:54.538276 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def9681c-2aa8-416a-88d7-2317b4343ee9 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:54.639789 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:54.644377 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def9681c-2aa8-416a-88d7-2317b4343ee9 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:54.687480 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84d06cc-613f-486f-ab34-9e46ed56400f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:54.688969 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d06cc-613f-486f-ab34-9e46ed56400f None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:54.689702 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d06cc-613f-486f-ab34-9e46ed56400f None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:54.690389 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d06cc-613f-486f-ab34-9e46ed56400f None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:54.692342 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84d06cc-613f-486f-ab34-9e46ed56400f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:54.693226 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84d06cc-613f-486f-ab34-9e46ed56400f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:54.695790 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84d06cc-613f-486f-ab34-9e46ed56400f None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:54.705517 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 464/923] 10.222.0.33 () {56 vars in 1122 bytes} [Fri Jul 15 19:15:54 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:54.717813 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661e5c86-1b74-4872-a32a-06ea519c6668 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:54.841185 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661e5c86-1b74-4872-a32a-06ea519c6668 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:54.876575 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cc12ef-c1e6-4969-8b69-318917b1050f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:54.877236 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cc12ef-c1e6-4969-8b69-318917b1050f None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:54.877520 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cc12ef-c1e6-4969-8b69-318917b1050f None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:54.877833 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cc12ef-c1e6-4969-8b69-318917b1050f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:54.878741 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cc12ef-c1e6-4969-8b69-318917b1050f None admin] RBAC: Adding query filter params (interface=public, service_id=e8870c0588ae49418f8532a6f8ec5491, region_id=RegionOne) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:54.879567 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cc12ef-c1e6-4969-8b69-318917b1050f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:54.880965 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cc12ef-c1e6-4969-8b69-318917b1050f None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:54.886957 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 460/924] 10.222.0.33 () {56 vars in 1251 bytes} [Fri Jul 15 19:15:54 2022] GET /identity/v3/endpoints?service_id=e8870c0588ae49418f8532a6f8ec5491&interface=public®ion_id=RegionOne => generated 194 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:54.892556 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84d06cc-613f-486f-ab34-9e46ed56400f None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:55.000259 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84d06cc-613f-486f-ab34-9e46ed56400f None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:55.040992 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a63a153-38f9-4647-938a-feb9ff36d6b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:55.041716 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a63a153-38f9-4647-938a-feb9ff36d6b1 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:55.042036 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a63a153-38f9-4647-938a-feb9ff36d6b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:55.042320 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a63a153-38f9-4647-938a-feb9ff36d6b1 None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:55.043014 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a63a153-38f9-4647-938a-feb9ff36d6b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:55.044379 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a63a153-38f9-4647-938a-feb9ff36d6b1 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:55.051887 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 465/925] 10.222.0.33 () {56 vars in 1087 bytes} [Fri Jul 15 19:15:54 2022] GET /identity/v3/services => generated 1497 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:15:55.159174 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229e1a38-0ec4-4bdc-a1c8-3e7487587170 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:55.159482 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229e1a38-0ec4-4bdc-a1c8-3e7487587170 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:55.159661 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229e1a38-0ec4-4bdc-a1c8-3e7487587170 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:55.161369 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 461/926] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:15:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:15:55.167119 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf075a6-7271-4321-9ad6-d2448f18da82 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:55.167439 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf075a6-7271-4321-9ad6-d2448f18da82 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:55.167674 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf075a6-7271-4321-9ad6-d2448f18da82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:55.186325 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-6bf075a6-7271-4321-9ad6-d2448f18da82 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:15:55.187568 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-6bf075a6-7271-4321-9ad6-d2448f18da82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:15:55.211970 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:55.260852 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 466/927] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:15:55 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:15:55.283639 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229e1a38-0ec4-4bdc-a1c8-3e7487587170 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:55.345654 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:55.390736 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229e1a38-0ec4-4bdc-a1c8-3e7487587170 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:55.406178 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e798eb1-583b-41b6-bcb5-6f7dd64e276c None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:55.407198 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e798eb1-583b-41b6-bcb5-6f7dd64e276c None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:55.407687 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e798eb1-583b-41b6-bcb5-6f7dd64e276c None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:55.411320 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 467/928] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:15:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:15:55.415697 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4436b238-cfd1-4120-9694-0ace5e4eafac None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:55.416235 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4436b238-cfd1-4120-9694-0ace5e4eafac None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:55.416725 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4436b238-cfd1-4120-9694-0ace5e4eafac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:55.427238 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ec8636-3e06-4a0a-a8d4-f09d6447d2e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:55.428053 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec8636-3e06-4a0a-a8d4-f09d6447d2e7 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:55.428334 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec8636-3e06-4a0a-a8d4-f09d6447d2e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:55.428718 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec8636-3e06-4a0a-a8d4-f09d6447d2e7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:55.435132 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-4436b238-cfd1-4120-9694-0ace5e4eafac None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:15:55.437001 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-4436b238-cfd1-4120-9694-0ace5e4eafac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:15:55.439536 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ec8636-3e06-4a0a-a8d4-f09d6447d2e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:15:55.448733 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-b4ec8636-3e06-4a0a-a8d4-f09d6447d2e7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 15 19:15:55.448733 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:55.448733 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:55.448733 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:55.448733 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:55.448733 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:55.448733 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:55.448733 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:55.448733 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:55.448733 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:55.448733 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:55.448733 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:55.448733 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:15:55.448733 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:15:55.448733 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:15:55.448733 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:15:55.449700 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:15:55.449700 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:15:55.449700 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:15:55.449700 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:15:55.449700 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:55.449700 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:55.449700 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:55.449700 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:55.449700 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:55.449700 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:55.449700 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:55.449700 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:55.449700 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:55.449700 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:55.449700 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:55.449700 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:55.451005 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:55.451005 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:55.451005 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:55.451005 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:55.451005 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:15:55.451005 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:15:55.451005 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:15:55.451005 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:15:55.451005 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:15:55.451005 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:15:55.451005 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 15 19:15:55.451005 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:55.451005 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 462/929] 10.222.0.33 () {56 vars in 1120 bytes} [Fri Jul 15 19:15:55 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:55.457810 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ec8636-3e06-4a0a-a8d4-f09d6447d2e7 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:55.549853 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 468/930] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:15:55 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 19:15:55.571140 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4436b238-cfd1-4120-9694-0ace5e4eafac None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:55.592164 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ec8636-3e06-4a0a-a8d4-f09d6447d2e7 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:55.617324 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:55.639780 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f711ac9-10f3-4561-b22b-336b12f0175e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:55.640998 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f711ac9-10f3-4561-b22b-336b12f0175e None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:55.641647 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f711ac9-10f3-4561-b22b-336b12f0175e None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:55.642189 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f711ac9-10f3-4561-b22b-336b12f0175e None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:55.643641 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f711ac9-10f3-4561-b22b-336b12f0175e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:55.644624 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f711ac9-10f3-4561-b22b-336b12f0175e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:55.646797 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f711ac9-10f3-4561-b22b-336b12f0175e None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:55.657841 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 463/931] 10.222.0.33 () {56 vars in 1115 bytes} [Fri Jul 15 19:15:55 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:15:55.663265 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f711ac9-10f3-4561-b22b-336b12f0175e None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:55.668622 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:55.685412 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4436b238-cfd1-4120-9694-0ace5e4eafac None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:55.723902 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56375b5-e9a4-46dd-b6f9-acf2446524cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:55.724944 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56375b5-e9a4-46dd-b6f9-acf2446524cb None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:55.725326 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56375b5-e9a4-46dd-b6f9-acf2446524cb None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:55.725959 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56375b5-e9a4-46dd-b6f9-acf2446524cb None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:55.726733 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56375b5-e9a4-46dd-b6f9-acf2446524cb None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:55.728299 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56375b5-e9a4-46dd-b6f9-acf2446524cb None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:55.742553 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-c56375b5-e9a4-46dd-b6f9-acf2446524cb None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 15 19:15:55.742553 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:55.742553 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 19:15:55.742553 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 19:15:55.742553 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 19:15:55.742553 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 19:15:55.742553 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 19:15:55.742553 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:15:55.742553 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 15 19:15:55.742553 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:55.742553 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 19:15:55.742553 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:55.742553 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:55.742553 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:55.742553 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:55.742553 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:55.743745 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:55.743745 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:55.743745 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:55.743745 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:55.743745 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:55.743745 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:55.743745 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:55.743745 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 19:15:55.743745 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 19:15:55.743745 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 19:15:55.743745 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 19:15:55.743745 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:55.743745 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:55.743745 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:55.743745 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:55.743745 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:55.744780 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:55.744780 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:55.744780 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:55.744780 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:55.744780 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:55.744780 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:55.744780 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:55.744780 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:55.744780 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:55.744780 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:55.744780 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:55.744780 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 19:15:55.744780 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:15:55.744780 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 15 19:15:55.744780 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:55.751639 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 469/932] 10.222.0.33 () {56 vars in 1132 bytes} [Fri Jul 15 19:15:55 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:15:55.753431 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56375b5-e9a4-46dd-b6f9-acf2446524cb None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:55.800684 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f711ac9-10f3-4561-b22b-336b12f0175e None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:55.848152 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7825fc35-a810-4b10-aa3d-ed8910da0bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:55.849508 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7825fc35-a810-4b10-aa3d-ed8910da0bce None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:55.850354 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7825fc35-a810-4b10-aa3d-ed8910da0bce None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:55.850972 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7825fc35-a810-4b10-aa3d-ed8910da0bce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:55.865929 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7825fc35-a810-4b10-aa3d-ed8910da0bce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:55.867455 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7825fc35-a810-4b10-aa3d-ed8910da0bce None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:55.873168 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56375b5-e9a4-46dd-b6f9-acf2446524cb None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:55.886673 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-7825fc35-a810-4b10-aa3d-ed8910da0bce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:55.886673 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:55.886673 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:55.886673 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:55.886673 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:55.886673 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:55.886673 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:55.886673 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:55.886673 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:55.886673 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:55.886673 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:55.886673 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:55.886673 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:15:55.886673 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:15:55.886673 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:15:55.887635 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:15:55.887635 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:55.887635 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:55.887635 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:55.887635 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:55.887635 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:55.887635 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:55.887635 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:55.887635 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:55.887635 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:55.887635 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:55.887635 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:55.887635 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:55.887635 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:55.887635 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:55.887635 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:55.887635 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:55.888945 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:15:55.888945 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:15:55.888945 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:55.888945 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:55.888945 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 464/933] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:15:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:55.893344 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7825fc35-a810-4b10-aa3d-ed8910da0bce None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:55.920042 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09ab32f-0cf6-45c3-8b8b-22456f94ac24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:55.920627 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09ab32f-0cf6-45c3-8b8b-22456f94ac24 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:55.920903 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09ab32f-0cf6-45c3-8b8b-22456f94ac24 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:55.921172 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09ab32f-0cf6-45c3-8b8b-22456f94ac24 None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:55.921910 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09ab32f-0cf6-45c3-8b8b-22456f94ac24 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:55.922235 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09ab32f-0cf6-45c3-8b8b-22456f94ac24 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:55.923554 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09ab32f-0cf6-45c3-8b8b-22456f94ac24 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:55.935193 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 470/934] 10.222.0.33 () {56 vars in 1126 bytes} [Fri Jul 15 19:15:55 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:55.937020 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09ab32f-0cf6-45c3-8b8b-22456f94ac24 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:55.940151 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:55.965716 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:56.012595 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7825fc35-a810-4b10-aa3d-ed8910da0bce None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:56.049752 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b81e218-27da-4481-9bdc-bc63677c5115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:56.050768 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b81e218-27da-4481-9bdc-bc63677c5115 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:56.050768 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b81e218-27da-4481-9bdc-bc63677c5115 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:56.051047 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b81e218-27da-4481-9bdc-bc63677c5115 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:56.051784 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b81e218-27da-4481-9bdc-bc63677c5115 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:56.052088 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b81e218-27da-4481-9bdc-bc63677c5115 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:56.053457 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b81e218-27da-4481-9bdc-bc63677c5115 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:56.071188 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 465/935] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:15:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:56.072105 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b81e218-27da-4481-9bdc-bc63677c5115 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:56.079843 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09ab32f-0cf6-45c3-8b8b-22456f94ac24 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:56.125394 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208ebef8-e61f-4e48-909a-1b012df7429f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:56.126741 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208ebef8-e61f-4e48-909a-1b012df7429f None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:56.127231 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208ebef8-e61f-4e48-909a-1b012df7429f None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:56.127746 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208ebef8-e61f-4e48-909a-1b012df7429f None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:56.128900 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208ebef8-e61f-4e48-909a-1b012df7429f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:56.130100 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208ebef8-e61f-4e48-909a-1b012df7429f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:56.132348 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208ebef8-e61f-4e48-909a-1b012df7429f None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:56.141108 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 471/936] 10.222.0.33 () {56 vars in 1126 bytes} [Fri Jul 15 19:15:55 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:56.150033 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208ebef8-e61f-4e48-909a-1b012df7429f None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:56.197450 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:56.204995 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b81e218-27da-4481-9bdc-bc63677c5115 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:56.246137 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765050ed-7021-4269-9b2e-5196e7933614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:56.247270 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765050ed-7021-4269-9b2e-5196e7933614 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:56.247831 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765050ed-7021-4269-9b2e-5196e7933614 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:56.248330 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765050ed-7021-4269-9b2e-5196e7933614 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:56.252791 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:56.258276 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765050ed-7021-4269-9b2e-5196e7933614 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:56.259971 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765050ed-7021-4269-9b2e-5196e7933614 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:56.275820 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-765050ed-7021-4269-9b2e-5196e7933614 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 15 19:15:56.275820 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:56.275820 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:56.275820 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:56.275820 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:56.275820 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:56.275820 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:56.275820 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:56.275820 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:56.275820 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:56.275820 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:56.275820 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:56.275820 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 19:15:56.275820 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 19:15:56.275820 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 19:15:56.275820 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 19:15:56.276829 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:56.276829 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:56.276829 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:15:56.276829 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:15:56.276829 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:15:56.276829 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:15:56.276829 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:56.276829 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:56.276829 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:56.276829 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:56.276829 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:56.276829 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:56.276829 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:56.276829 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:56.276829 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:56.276829 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:56.276829 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:56.276829 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:56.277814 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:56.277814 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:56.277814 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 19:15:56.277814 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 19:15:56.277814 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 19:15:56.277814 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 19:15:56.277814 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 19:15:56.277814 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 19:15:56.277814 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 15 19:15:56.277814 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:56.278372 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208ebef8-e61f-4e48-909a-1b012df7429f None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:56.280582 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 466/937] 10.222.0.33 () {56 vars in 1117 bytes} [Fri Jul 15 19:15:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:56.286711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765050ed-7021-4269-9b2e-5196e7933614 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:56.317672 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4cb5d9-2c68-45bc-a210-38b1a46f4168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:56.318766 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4cb5d9-2c68-45bc-a210-38b1a46f4168 None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:56.319262 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4cb5d9-2c68-45bc-a210-38b1a46f4168 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:56.319801 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4cb5d9-2c68-45bc-a210-38b1a46f4168 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:56.320880 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4cb5d9-2c68-45bc-a210-38b1a46f4168 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:56.322510 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4cb5d9-2c68-45bc-a210-38b1a46f4168 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:56.399360 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765050ed-7021-4269-9b2e-5196e7933614 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:56.424256 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 472/938] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 15 19:15:56 2022] POST /identity/v3/endpoints => generated 347 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:15:56.457112 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759c765a-aa1a-43f0-ac8c-8a9ab0010c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:56.458237 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759c765a-aa1a-43f0-ac8c-8a9ab0010c12 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:56.458753 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759c765a-aa1a-43f0-ac8c-8a9ab0010c12 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:56.459253 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759c765a-aa1a-43f0-ac8c-8a9ab0010c12 None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:56.460458 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759c765a-aa1a-43f0-ac8c-8a9ab0010c12 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:56.461017 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759c765a-aa1a-43f0-ac8c-8a9ab0010c12 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:56.462609 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835de8d1-0120-491d-9e97-c2b543016814 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:56.462968 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835de8d1-0120-491d-9e97-c2b543016814 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:56.463261 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759c765a-aa1a-43f0-ac8c-8a9ab0010c12 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:56.463530 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835de8d1-0120-491d-9e97-c2b543016814 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:56.465378 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 473/939] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:15:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 19:15:56.471357 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d600d2c-894e-4bc8-8098-ea3ed2de1aca None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:56.471738 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d600d2c-894e-4bc8-8098-ea3ed2de1aca None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:56.474861 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d600d2c-894e-4bc8-8098-ea3ed2de1aca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:56.491233 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 467/940] 10.222.0.33 () {56 vars in 1149 bytes} [Fri Jul 15 19:15:56 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:56.494450 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-6d600d2c-894e-4bc8-8098-ea3ed2de1aca None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:15:56.495720 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-6d600d2c-894e-4bc8-8098-ea3ed2de1aca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:15:56.496639 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759c765a-aa1a-43f0-ac8c-8a9ab0010c12 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:56.508979 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:56.589812 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 474/941] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:15:56 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:15:56.608247 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d600d2c-894e-4bc8-8098-ea3ed2de1aca None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:56.633478 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759c765a-aa1a-43f0-ac8c-8a9ab0010c12 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:56.680134 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a97cc5-0217-460d-8c66-5ac274aea3e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:56.681039 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a97cc5-0217-460d-8c66-5ac274aea3e0 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:56.681467 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a97cc5-0217-460d-8c66-5ac274aea3e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:56.681840 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a97cc5-0217-460d-8c66-5ac274aea3e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:56.688754 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:56.694898 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a97cc5-0217-460d-8c66-5ac274aea3e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:56.696616 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a97cc5-0217-460d-8c66-5ac274aea3e0 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:56.718798 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-32a97cc5-0217-460d-8c66-5ac274aea3e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:56.718798 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:56.718798 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:56.718798 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:56.718798 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:56.718798 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:56.718798 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:56.718798 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:56.718798 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:56.718798 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:56.718798 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:56.718798 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:56.718798 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:15:56.718798 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:15:56.718798 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:15:56.719895 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:15:56.719895 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:56.719895 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:56.719895 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:56.719895 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:56.719895 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:56.719895 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:56.719895 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:56.719895 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:56.719895 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:56.719895 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:56.719895 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:56.719895 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:56.719895 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:56.719895 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:56.719895 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:56.719895 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:56.721018 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:15:56.721018 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:15:56.721018 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:56.721018 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:56.724306 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 468/942] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:15:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:15:56.729449 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a97cc5-0217-460d-8c66-5ac274aea3e0 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:56.740710 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d600d2c-894e-4bc8-8098-ea3ed2de1aca None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:56.787260 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cfdef6-cb88-498d-b2ab-284e18fd4159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:56.788543 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cfdef6-cb88-498d-b2ab-284e18fd4159 None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:56.789449 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cfdef6-cb88-498d-b2ab-284e18fd4159 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:56.789974 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cfdef6-cb88-498d-b2ab-284e18fd4159 None admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:56.791080 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cfdef6-cb88-498d-b2ab-284e18fd4159 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:56.793528 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cfdef6-cb88-498d-b2ab-284e18fd4159 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:56.795928 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: WARNING keystone.server.flask.common [None req-f5cfdef6-cb88-498d-b2ab-284e18fd4159 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 15 19:15:56.798047 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:56.842312 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.sql.core [None req-f5cfdef6-cb88-498d-b2ab-284e18fd4159 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 15 19:15:56.842312 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 15 19:15:56.842312 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [parameters: {'id': '48e915a26b0f4a2a8bce2c74265c895b', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 15 19:15:56.842312 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80268) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 15 19:15:56.843479 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a97cc5-0217-460d-8c66-5ac274aea3e0 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:56.852799 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-f5cfdef6-cb88-498d-b2ab-284e18fd4159 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 48e915a26b0f4a2a8bce2c74265c895b.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 48e915a26b0f4a2a8bce2c74265c895b. Jul 15 19:15:56.852799 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:56.852799 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 15 19:15:56.852799 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 15 19:15:56.852799 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 15 19:15:56.852799 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 15 19:15:56.852799 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 15 19:15:56.852799 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application result = self._query(query) Jul 15 19:15:56.852799 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 15 19:15:56.852799 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application conn.query(q) Jul 15 19:15:56.852799 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 15 19:15:56.852799 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 15 19:15:56.852799 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 15 19:15:56.853651 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application result.read() Jul 15 19:15:56.853651 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 15 19:15:56.853651 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 15 19:15:56.853651 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 15 19:15:56.853651 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 15 19:15:56.853651 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 15 19:15:56.853651 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 15 19:15:56.853651 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 15 19:15:56.853651 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 15 19:15:56.853651 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 15 19:15:56.853651 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:56.853651 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 15 19:15:56.853651 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:56.853651 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:56.853651 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 15 19:15:56.853651 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 15 19:15:56.853651 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 15 19:15:56.853651 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 15 19:15:56.854653 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 15 19:15:56.854653 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application next(self.gen) Jul 15 19:15:56.854653 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 15 19:15:56.854653 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application yield resource Jul 15 19:15:56.854653 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 15 19:15:56.854653 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application next(self.gen) Jul 15 19:15:56.854653 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 15 19:15:56.854653 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self.session.rollback() Jul 15 19:15:56.854653 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 15 19:15:56.854653 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self.force_reraise() Jul 15 19:15:56.854653 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 15 19:15:56.854653 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise self.value Jul 15 19:15:56.854653 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 15 19:15:56.854653 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 15 19:15:56.854653 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 15 19:15:56.854653 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application session.commit() Jul 15 19:15:56.854653 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 15 19:15:56.855551 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 15 19:15:56.855551 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 15 19:15:56.855551 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._prepare_impl() Jul 15 19:15:56.855551 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 15 19:15:56.855551 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self.session.flush() Jul 15 19:15:56.855551 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 15 19:15:56.855551 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._flush(objects) Jul 15 19:15:56.855551 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 15 19:15:56.855551 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 15 19:15:56.855551 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 15 19:15:56.855551 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application compat.raise_( Jul 15 19:15:56.855551 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 15 19:15:56.855551 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception Jul 15 19:15:56.855551 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 15 19:15:56.855551 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application flush_context.execute() Jul 15 19:15:56.855551 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 15 19:15:56.855551 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rec.execute(self) Jul 15 19:15:56.856447 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 15 19:15:56.856447 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 15 19:15:56.856447 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 15 19:15:56.856447 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 15 19:15:56.856447 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 15 19:15:56.856447 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 15 19:15:56.856447 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 15 19:15:56.856447 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 15 19:15:56.856447 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 15 19:15:56.856447 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 15 19:15:56.856447 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 15 19:15:56.856447 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 15 19:15:56.856447 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 15 19:15:56.856447 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 15 19:15:56.856447 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 15 19:15:56.857249 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 15 19:15:56.857249 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 15 19:15:56.857249 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception Jul 15 19:15:56.857249 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 15 19:15:56.857249 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 15 19:15:56.857249 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 15 19:15:56.857249 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 15 19:15:56.857249 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 15 19:15:56.857249 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application result = self._query(query) Jul 15 19:15:56.857249 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 15 19:15:56.857249 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application conn.query(q) Jul 15 19:15:56.857249 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 15 19:15:56.857249 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 15 19:15:56.857249 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 15 19:15:56.857249 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application result.read() Jul 15 19:15:56.857249 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 15 19:15:56.857249 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 15 19:15:56.859512 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 15 19:15:56.859512 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 15 19:15:56.859512 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 15 19:15:56.859512 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 15 19:15:56.859512 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 15 19:15:56.859512 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 15 19:15:56.859512 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 15 19:15:56.859512 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 15 19:15:56.859512 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [parameters: {'id': '48e915a26b0f4a2a8bce2c74265c895b', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 15 19:15:56.859512 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 15 19:15:56.859512 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:56.859512 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 19:15:56.859512 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:56.859512 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:56.859512 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 15 19:15:56.861264 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 15 19:15:56.861264 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 15 19:15:56.861264 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 15 19:15:56.861264 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 15 19:15:56.861264 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:56.861264 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 19:15:56.861264 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:56.861264 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:56.861264 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:56.861264 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:56.861264 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:56.861264 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:56.861264 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:56.861264 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:56.861264 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:56.861264 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:56.861264 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:56.867687 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:56.867687 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 15 19:15:56.867687 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 15 19:15:56.867687 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:56.867687 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:56.867687 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 15 19:15:56.867687 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 15 19:15:56.867687 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 48e915a26b0f4a2a8bce2c74265c895b. Jul 15 19:15:56.867687 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:56.867687 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 475/943] 10.222.0.33 () {60 vars in 1140 bytes} [Fri Jul 15 19:15:56 2022] POST /identity/v3/projects => generated 262 bytes in 247 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:15:56.871572 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cfdef6-cb88-498d-b2ab-284e18fd4159 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:56.885811 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863cbfd1-b8cb-45d4-82ef-fefff91cf44f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:56.886497 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863cbfd1-b8cb-45d4-82ef-fefff91cf44f None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:56.886779 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863cbfd1-b8cb-45d4-82ef-fefff91cf44f None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:56.887055 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863cbfd1-b8cb-45d4-82ef-fefff91cf44f None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:56.887782 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863cbfd1-b8cb-45d4-82ef-fefff91cf44f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:56.888088 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863cbfd1-b8cb-45d4-82ef-fefff91cf44f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:56.889456 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863cbfd1-b8cb-45d4-82ef-fefff91cf44f None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:56.906868 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 469/944] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:15:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:56.908436 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863cbfd1-b8cb-45d4-82ef-fefff91cf44f None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:56.985166 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cfdef6-cb88-498d-b2ab-284e18fd4159 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:57.007736 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:57.016798 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863cbfd1-b8cb-45d4-82ef-fefff91cf44f None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:57.026191 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b93d14-3430-4980-b9eb-4036757244eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:57.027442 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b93d14-3430-4980-b9eb-4036757244eb None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:57.027953 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b93d14-3430-4980-b9eb-4036757244eb None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:57.028504 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b93d14-3430-4980-b9eb-4036757244eb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:57.036026 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:57.037591 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b93d14-3430-4980-b9eb-4036757244eb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:57.039463 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b93d14-3430-4980-b9eb-4036757244eb None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:57.053574 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591b236f-62e2-4569-a8d8-6365ad1b6be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:57.055707 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b236f-62e2-4569-a8d8-6365ad1b6be0 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:57.055819 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-b0b93d14-3430-4980-b9eb-4036757244eb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:15:57.055819 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:57.055819 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:57.055819 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:57.055819 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:57.055819 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:57.055819 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:57.055819 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:57.055819 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:57.055819 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:57.055819 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:57.055819 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:57.055819 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 19:15:57.055819 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 19:15:57.055819 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 19:15:57.056816 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 19:15:57.056816 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:57.056816 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:57.056816 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:57.056816 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:57.056816 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:57.056816 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:57.056816 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:57.056816 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:57.056816 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:57.056816 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:57.056816 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:57.056816 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:57.056816 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:57.056816 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:57.056816 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:57.056816 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:57.057929 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 19:15:57.057929 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 19:15:57.057929 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:15:57.057929 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:15:57.057929 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:15:57.057929 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:15:57.057929 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:15:57.057929 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:57.058479 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b236f-62e2-4569-a8d8-6365ad1b6be0 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:57.058479 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b236f-62e2-4569-a8d8-6365ad1b6be0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:57.058629 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 476/945] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 15 19:15:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:57.065561 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b93d14-3430-4980-b9eb-4036757244eb None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:57.071029 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591b236f-62e2-4569-a8d8-6365ad1b6be0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:57.072675 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591b236f-62e2-4569-a8d8-6365ad1b6be0 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:57.088253 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-591b236f-62e2-4569-a8d8-6365ad1b6be0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:15:57.088253 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:57.088253 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:57.088253 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:57.088253 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:57.088253 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:57.088253 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:57.088253 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:57.088253 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:57.088253 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:57.088253 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:57.088253 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:57.088253 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 19:15:57.088253 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 19:15:57.088253 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 19:15:57.089384 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 19:15:57.089384 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:57.089384 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:57.089384 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:57.089384 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:57.089384 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:57.089384 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:57.089384 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:57.089384 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:57.089384 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:57.089384 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:57.089384 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:57.089384 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:57.089384 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:57.089384 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:57.089384 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:57.089384 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:57.090710 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 19:15:57.090710 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 19:15:57.090710 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:15:57.090710 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:15:57.090710 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:15:57.090710 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:15:57.090710 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:15:57.090710 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:57.094080 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 470/946] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 15 19:15:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:15:57.099946 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591b236f-62e2-4569-a8d8-6365ad1b6be0 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:57.187314 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b93d14-3430-4980-b9eb-4036757244eb None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:57.224044 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be99cd8-9569-4cae-b19e-9b7ebf8b4df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:57.224706 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be99cd8-9569-4cae-b19e-9b7ebf8b4df3 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:57.224986 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be99cd8-9569-4cae-b19e-9b7ebf8b4df3 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:57.225258 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be99cd8-9569-4cae-b19e-9b7ebf8b4df3 None admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:57.225388 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591b236f-62e2-4569-a8d8-6365ad1b6be0 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:57.226040 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be99cd8-9569-4cae-b19e-9b7ebf8b4df3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:57.226345 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be99cd8-9569-4cae-b19e-9b7ebf8b4df3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:57.227771 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be99cd8-9569-4cae-b19e-9b7ebf8b4df3 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:57.250421 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 477/947] 10.222.0.33 () {56 vars in 1112 bytes} [Fri Jul 15 19:15:57 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:15:57.252017 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeb403b-7a29-4088-a200-aaa6ca93b0e7 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:57.252488 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeb403b-7a29-4088-a200-aaa6ca93b0e7 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:57.252920 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeb403b-7a29-4088-a200-aaa6ca93b0e7 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:57.257799 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 478/948] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:15:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:15:57.260685 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51af653d-3c6e-4715-98e2-82bfd6b452c7 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:57.260980 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51af653d-3c6e-4715-98e2-82bfd6b452c7 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:57.261266 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51af653d-3c6e-4715-98e2-82bfd6b452c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:57.277130 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aebf0f-cc34-40bb-9909-9b4ec6098d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:57.278614 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aebf0f-cc34-40bb-9909-9b4ec6098d3f None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:57.279120 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aebf0f-cc34-40bb-9909-9b4ec6098d3f None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:57.279623 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aebf0f-cc34-40bb-9909-9b4ec6098d3f None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:57.280671 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65aebf0f-cc34-40bb-9909-9b4ec6098d3f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:57.281232 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65aebf0f-cc34-40bb-9909-9b4ec6098d3f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:57.282240 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-51af653d-3c6e-4715-98e2-82bfd6b452c7 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:15:57.283230 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65aebf0f-cc34-40bb-9909-9b4ec6098d3f None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:57.283826 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-51af653d-3c6e-4715-98e2-82bfd6b452c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:15:57.300992 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 471/949] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jul 15 19:15:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:15:57.306826 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aebf0f-cc34-40bb-9909-9b4ec6098d3f None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:57.333313 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:57.362231 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:57.381947 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 479/950] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:15:57 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 19:15:57.402700 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51af653d-3c6e-4715-98e2-82bfd6b452c7 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:57.443461 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aebf0f-cc34-40bb-9909-9b4ec6098d3f None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:57.488256 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be8a514-49d5-41ba-923a-eff79406a933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:57.489326 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be8a514-49d5-41ba-923a-eff79406a933 None admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:57.489691 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be8a514-49d5-41ba-923a-eff79406a933 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:57.490096 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be8a514-49d5-41ba-923a-eff79406a933 None admin] PATH_INFO: `/v3/projects/55a756ccaef14cb49b26b0d68dd0b466/users/3d893c14620b4a129842062f14a68a78/roles/c1dc7198be2d43d3aa44d16107d9a959` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:57.492439 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be8a514-49d5-41ba-923a-eff79406a933 None admin] RBAC: Authorizing `identity:create_grant(project_id=55a756ccaef14cb49b26b0d68dd0b466, user_id=3d893c14620b4a129842062f14a68a78, role_id=c1dc7198be2d43d3aa44d16107d9a959)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:57.493942 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be8a514-49d5-41ba-923a-eff79406a933 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:57.522204 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51af653d-3c6e-4715-98e2-82bfd6b452c7 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:57.557983 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd17f97-1d98-490d-869b-76aa0ac43be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:57.558794 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd17f97-1d98-490d-869b-76aa0ac43be8 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:57.559078 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd17f97-1d98-490d-869b-76aa0ac43be8 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:57.559353 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd17f97-1d98-490d-869b-76aa0ac43be8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:57.560014 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd17f97-1d98-490d-869b-76aa0ac43be8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:57.568845 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 472/951] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jul 15 19:15:57 2022] PUT /identity/v3/projects/55a756ccaef14cb49b26b0d68dd0b466/users/3d893c14620b4a129842062f14a68a78/roles/c1dc7198be2d43d3aa44d16107d9a959 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:15:57.571934 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd17f97-1d98-490d-869b-76aa0ac43be8 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:57.592076 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-1cd17f97-1d98-490d-869b-76aa0ac43be8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 19:15:57.592076 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:57.592076 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 19:15:57.592076 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 19:15:57.592076 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 19:15:57.592076 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 19:15:57.592076 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 19:15:57.592076 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:15:57.592076 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 19:15:57.592076 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:57.592076 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 19:15:57.592076 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:57.592076 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:57.592076 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:57.592076 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:57.592076 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:57.592076 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:57.593552 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:57.593552 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:57.593552 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:57.593552 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:57.593552 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:57.593552 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:57.593552 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 19:15:57.593552 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 19:15:57.593552 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 19:15:57.593552 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 19:15:57.593552 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:57.593552 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:57.593552 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:57.593552 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:57.593552 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:57.593552 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:57.594727 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:57.594727 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:57.594727 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:57.594727 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:57.594727 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:57.594727 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:57.594727 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:57.594727 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:57.594727 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:57.594727 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:57.594727 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 19:15:57.594727 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:15:57.594727 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 19:15:57.594727 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:57.599178 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 480/952] 10.222.0.33 () {56 vars in 1126 bytes} [Fri Jul 15 19:15:57 2022] GET /identity/v3/services/object-store => generated 93 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:57.606602 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be8a514-49d5-41ba-923a-eff79406a933 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:57.717784 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:57.722944 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be8a514-49d5-41ba-923a-eff79406a933 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:57.761486 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aafe3fd-7214-4531-9dd4-15e60bd382e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:57.762196 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aafe3fd-7214-4531-9dd4-15e60bd382e5 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:57.762529 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aafe3fd-7214-4531-9dd4-15e60bd382e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:57.762805 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aafe3fd-7214-4531-9dd4-15e60bd382e5 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:57.763588 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aafe3fd-7214-4531-9dd4-15e60bd382e5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:57.763888 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aafe3fd-7214-4531-9dd4-15e60bd382e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:57.765349 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aafe3fd-7214-4531-9dd4-15e60bd382e5 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:57.771274 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 473/953] 10.222.0.33 () {56 vars in 1122 bytes} [Fri Jul 15 19:15:57 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:57.776908 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd17f97-1d98-490d-869b-76aa0ac43be8 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:57.820862 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:57.892155 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd17f97-1d98-490d-869b-76aa0ac43be8 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:57.928589 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e9485e-55db-4dbe-8a78-5a299833d264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:57.929258 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e9485e-55db-4dbe-8a78-5a299833d264 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:57.929541 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e9485e-55db-4dbe-8a78-5a299833d264 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:57.929849 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e9485e-55db-4dbe-8a78-5a299833d264 None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:57.930563 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e9485e-55db-4dbe-8a78-5a299833d264 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:57.930870 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e9485e-55db-4dbe-8a78-5a299833d264 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:57.932240 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e9485e-55db-4dbe-8a78-5a299833d264 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:57.937696 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 481/954] 10.222.0.33 () {56 vars in 1122 bytes} [Fri Jul 15 19:15:57 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:57.943338 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aafe3fd-7214-4531-9dd4-15e60bd382e5 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:58.045574 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aafe3fd-7214-4531-9dd4-15e60bd382e5 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:58.080883 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2782e0af-ff7a-42bd-aa35-5d6478ed7317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:58.082512 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2782e0af-ff7a-42bd-aa35-5d6478ed7317 None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:58.083001 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2782e0af-ff7a-42bd-aa35-5d6478ed7317 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:58.083472 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2782e0af-ff7a-42bd-aa35-5d6478ed7317 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:58.084402 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2782e0af-ff7a-42bd-aa35-5d6478ed7317 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:58.086007 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2782e0af-ff7a-42bd-aa35-5d6478ed7317 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:58.164103 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 474/955] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 15 19:15:57 2022] POST /identity/v3/endpoints => generated 349 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:15:58.648276 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36178477-ff45-4622-b633-a7facc564de5 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:58.650043 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36178477-ff45-4622-b633-a7facc564de5 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:58.650043 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36178477-ff45-4622-b633-a7facc564de5 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:58.650395 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 482/956] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:15:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:15:58.655904 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a811f2-75a0-4623-99f5-3d00a71c8edd None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:58.656714 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a811f2-75a0-4623-99f5-3d00a71c8edd None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:58.657199 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a811f2-75a0-4623-99f5-3d00a71c8edd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:58.674667 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-f3a811f2-75a0-4623-99f5-3d00a71c8edd None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:15:58.676721 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-f3a811f2-75a0-4623-99f5-3d00a71c8edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:15:58.690697 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:58.753343 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 475/957] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:15:58 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:15:58.781047 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36178477-ff45-4622-b633-a7facc564de5 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:58.882581 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:58.887102 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36178477-ff45-4622-b633-a7facc564de5 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:58.922916 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b538df-3a93-4efc-bcfd-f8d20407d496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:58.924067 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b538df-3a93-4efc-bcfd-f8d20407d496 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:58.924852 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b538df-3a93-4efc-bcfd-f8d20407d496 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:58.925129 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b538df-3a93-4efc-bcfd-f8d20407d496 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:58.925835 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b538df-3a93-4efc-bcfd-f8d20407d496 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:58.927215 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b538df-3a93-4efc-bcfd-f8d20407d496 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:58.939433 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-c1b538df-3a93-4efc-bcfd-f8d20407d496 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 15 19:15:58.939433 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:58.939433 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 19:15:58.939433 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 19:15:58.939433 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 19:15:58.939433 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 19:15:58.939433 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 19:15:58.939433 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:15:58.939433 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 15 19:15:58.939433 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:58.939433 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 19:15:58.939433 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:58.939433 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:58.939433 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:58.939433 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:58.939433 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:58.939433 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:58.940562 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:58.940562 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:58.940562 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:58.940562 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:58.940562 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:58.940562 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:58.940562 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 19:15:58.940562 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 19:15:58.940562 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 19:15:58.940562 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 19:15:58.940562 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:58.940562 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:58.940562 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:58.940562 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:58.940562 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:58.940562 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:58.941487 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:58.941487 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:58.941487 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:58.941487 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:58.941487 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:58.941487 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:58.941487 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:58.941487 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:58.941487 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:58.941487 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:58.941487 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 19:15:58.941487 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:15:58.941487 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 15 19:15:58.941487 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:15:58.941487 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 483/958] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 15 19:15:58 2022] GET /identity/v3/services/compute => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:58.946688 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a811f2-75a0-4623-99f5-3d00a71c8edd None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:59.054187 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a811f2-75a0-4623-99f5-3d00a71c8edd None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:59.089826 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2ae685-2aab-4dab-92f0-d47235194b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:59.090868 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2ae685-2aab-4dab-92f0-d47235194b9c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:59.091391 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2ae685-2aab-4dab-92f0-d47235194b9c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:59.091876 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2ae685-2aab-4dab-92f0-d47235194b9c None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:59.092790 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2ae685-2aab-4dab-92f0-d47235194b9c None admin] RBAC: Adding query filter params (name=compute) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:59.093319 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2ae685-2aab-4dab-92f0-d47235194b9c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:59.094980 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2ae685-2aab-4dab-92f0-d47235194b9c None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:59.101080 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 476/959] 10.222.0.33 () {56 vars in 1112 bytes} [Fri Jul 15 19:15:58 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:59.109785 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b538df-3a93-4efc-bcfd-f8d20407d496 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:59.207869 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b538df-3a93-4efc-bcfd-f8d20407d496 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:59.242004 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d56ded-35ed-4e3c-9723-c3b33fb46b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:59.242687 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d56ded-35ed-4e3c-9723-c3b33fb46b62 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:59.242999 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d56ded-35ed-4e3c-9723-c3b33fb46b62 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:59.243276 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d56ded-35ed-4e3c-9723-c3b33fb46b62 None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:59.243984 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d56ded-35ed-4e3c-9723-c3b33fb46b62 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:59.244290 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d56ded-35ed-4e3c-9723-c3b33fb46b62 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:59.245649 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d56ded-35ed-4e3c-9723-c3b33fb46b62 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:59.250670 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 484/960] 10.222.0.33 () {56 vars in 1112 bytes} [Fri Jul 15 19:15:59 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:59.481579 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ab6130-3aa2-4bca-a3d0-d7e22d08dca0 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:59.481952 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ab6130-3aa2-4bca-a3d0-d7e22d08dca0 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:59.482059 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ab6130-3aa2-4bca-a3d0-d7e22d08dca0 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:59.483677 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 477/961] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:15:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:15:59.489527 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7d39c6-49cc-4921-8a19-34aef329d5a1 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:59.489839 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7d39c6-49cc-4921-8a19-34aef329d5a1 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:59.490119 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7d39c6-49cc-4921-8a19-34aef329d5a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:59.508324 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-5e7d39c6-49cc-4921-8a19-34aef329d5a1 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:15:59.509847 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-5e7d39c6-49cc-4921-8a19-34aef329d5a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:15:59.532043 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:59.582898 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 485/962] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:15:59 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:15:59.608193 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ab6130-3aa2-4bca-a3d0-d7e22d08dca0 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:59.620102 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:59.676572 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08668983-0522-4ef2-a3f2-59c8fdd40887 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:59.677084 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08668983-0522-4ef2-a3f2-59c8fdd40887 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:59.677195 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08668983-0522-4ef2-a3f2-59c8fdd40887 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:59.679589 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 486/963] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:15:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 19:15:59.689505 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c493c6f-5314-4e91-8aa3-5b730155a1ac None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:59.689958 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c493c6f-5314-4e91-8aa3-5b730155a1ac None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:59.690388 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c493c6f-5314-4e91-8aa3-5b730155a1ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:59.708260 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-0c493c6f-5314-4e91-8aa3-5b730155a1ac None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:15:59.709610 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-0c493c6f-5314-4e91-8aa3-5b730155a1ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:15:59.723954 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ab6130-3aa2-4bca-a3d0-d7e22d08dca0 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:59.760239 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8427f3e-8137-4208-9d11-f82d2048b064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:59.761190 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8427f3e-8137-4208-9d11-f82d2048b064 None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:59.761639 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8427f3e-8137-4208-9d11-f82d2048b064 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:59.762093 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8427f3e-8137-4208-9d11-f82d2048b064 None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:59.762942 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8427f3e-8137-4208-9d11-f82d2048b064 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:59.764523 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8427f3e-8137-4208-9d11-f82d2048b064 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:59.766016 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: WARNING keystone.server.flask.common [None req-c8427f3e-8137-4208-9d11-f82d2048b064 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 15 19:15:59.785189 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 487/964] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:15:59 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 19:15:59.808282 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c493c6f-5314-4e91-8aa3-5b730155a1ac None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:59.813410 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.sql.core [None req-c8427f3e-8137-4208-9d11-f82d2048b064 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 15 19:15:59.813410 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 15 19:15:59.813410 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [parameters: {'user_id': 'b6ca8e3c7cbf4e6ea4352549279c8b8c', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 15 19:15:59.813410 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80269) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 15 19:15:59.833780 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-c8427f3e-8137-4208-9d11-f82d2048b064 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 15 19:15:59.833780 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:59.833780 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 15 19:15:59.833780 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 15 19:15:59.833780 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 15 19:15:59.833780 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 15 19:15:59.833780 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 15 19:15:59.833780 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application result = self._query(query) Jul 15 19:15:59.833780 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 15 19:15:59.833780 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application conn.query(q) Jul 15 19:15:59.833780 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 15 19:15:59.833780 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 15 19:15:59.833780 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 15 19:15:59.833780 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application result.read() Jul 15 19:15:59.833780 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 15 19:15:59.834753 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 15 19:15:59.834753 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 15 19:15:59.834753 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 15 19:15:59.834753 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 15 19:15:59.834753 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 15 19:15:59.834753 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 15 19:15:59.834753 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 15 19:15:59.834753 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 15 19:15:59.834753 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:59.834753 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 15 19:15:59.834753 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:59.834753 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:59.834753 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 15 19:15:59.834753 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 15 19:15:59.834753 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 15 19:15:59.834753 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 15 19:15:59.834753 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 15 19:15:59.834753 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application next(self.gen) Jul 15 19:15:59.835717 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 15 19:15:59.835717 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application yield resource Jul 15 19:15:59.835717 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 15 19:15:59.835717 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application next(self.gen) Jul 15 19:15:59.835717 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 15 19:15:59.835717 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self.session.rollback() Jul 15 19:15:59.835717 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 15 19:15:59.835717 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self.force_reraise() Jul 15 19:15:59.835717 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 15 19:15:59.835717 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise self.value Jul 15 19:15:59.835717 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 15 19:15:59.835717 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 15 19:15:59.835717 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 15 19:15:59.835717 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application session.commit() Jul 15 19:15:59.835717 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 15 19:15:59.835717 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 15 19:15:59.835717 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 15 19:15:59.836632 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._prepare_impl() Jul 15 19:15:59.836632 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 15 19:15:59.836632 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self.session.flush() Jul 15 19:15:59.836632 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 15 19:15:59.836632 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._flush(objects) Jul 15 19:15:59.836632 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 15 19:15:59.836632 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 15 19:15:59.836632 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 15 19:15:59.836632 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application compat.raise_( Jul 15 19:15:59.836632 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 15 19:15:59.836632 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception Jul 15 19:15:59.836632 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 15 19:15:59.836632 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application flush_context.execute() Jul 15 19:15:59.836632 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 15 19:15:59.836632 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rec.execute(self) Jul 15 19:15:59.836632 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 15 19:15:59.836632 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 15 19:15:59.837557 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 15 19:15:59.837557 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 15 19:15:59.837557 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 15 19:15:59.837557 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 15 19:15:59.837557 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 15 19:15:59.837557 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 15 19:15:59.837557 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 15 19:15:59.837557 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 15 19:15:59.837557 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 15 19:15:59.837557 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 15 19:15:59.837557 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 15 19:15:59.837557 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 15 19:15:59.837557 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 15 19:15:59.837557 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 15 19:15:59.837557 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 15 19:15:59.837557 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception Jul 15 19:15:59.838460 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 15 19:15:59.838460 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 15 19:15:59.838460 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 15 19:15:59.838460 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 15 19:15:59.838460 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 15 19:15:59.838460 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application result = self._query(query) Jul 15 19:15:59.838460 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 15 19:15:59.838460 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application conn.query(q) Jul 15 19:15:59.838460 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 15 19:15:59.838460 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 15 19:15:59.838460 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 15 19:15:59.838460 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application result.read() Jul 15 19:15:59.838460 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 15 19:15:59.838460 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 15 19:15:59.838460 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 15 19:15:59.838460 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 15 19:15:59.839312 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 15 19:15:59.839312 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 15 19:15:59.839312 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 15 19:15:59.839312 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 15 19:15:59.839312 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 15 19:15:59.839312 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 15 19:15:59.839312 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [parameters: {'user_id': 'b6ca8e3c7cbf4e6ea4352549279c8b8c', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 15 19:15:59.839312 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 15 19:15:59.839312 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:59.839312 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 19:15:59.839312 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:59.839312 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:59.839312 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:59.839312 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:59.839312 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:59.840118 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:59.840118 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:59.840118 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:59.840118 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:59.840118 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:59.840118 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:59.840118 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:59.840118 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 15 19:15:59.840118 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 15 19:15:59.840118 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:59.840118 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:59.840118 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:15:59.840118 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:15:59.840118 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:15:59.840118 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:15:59.840118 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 15 19:15:59.840118 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 15 19:15:59.841111 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 15 19:15:59.841111 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 15 19:15:59.841111 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 15 19:15:59.841111 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 15 19:15:59.841111 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 15 19:15:59.841111 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:15:59.847004 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 478/965] 10.222.0.33 () {60 vars in 1131 bytes} [Fri Jul 15 19:15:59 2022] POST /identity/v3/users => generated 160 bytes in 239 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:59.852685 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8427f3e-8137-4208-9d11-f82d2048b064 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:59.928760 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:59.948151 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c493c6f-5314-4e91-8aa3-5b730155a1ac None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:59.954379 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:59.995014 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a331f1cc-376e-43d8-827c-b2695bd4b00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:59.995846 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a331f1cc-376e-43d8-827c-b2695bd4b00a None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:59.996133 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a331f1cc-376e-43d8-827c-b2695bd4b00a None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:59.996410 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a331f1cc-376e-43d8-827c-b2695bd4b00a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:59.998127 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8427f3e-8137-4208-9d11-f82d2048b064 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:00.012072 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a331f1cc-376e-43d8-827c-b2695bd4b00a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:16:00.024798 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-a331f1cc-376e-43d8-827c-b2695bd4b00a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 15 19:16:00.024798 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:16:00.024798 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:16:00.024798 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:16:00.024798 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:16:00.024798 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:16:00.024798 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:16:00.024798 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:16:00.024798 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:16:00.024798 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:16:00.024798 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:16:00.024798 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:16:00.024798 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:16:00.024798 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:16:00.024798 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:16:00.024798 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:16:00.025807 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:16:00.025807 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:16:00.025807 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:16:00.025807 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:16:00.025807 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:16:00.025807 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:16:00.025807 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:16:00.025807 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:16:00.025807 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:16:00.025807 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:16:00.025807 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:16:00.025807 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:16:00.025807 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:16:00.025807 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:16:00.025807 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:16:00.025807 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:16:00.026708 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:16:00.026708 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:16:00.026708 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:16:00.026708 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:16:00.026708 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:16:00.026708 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:16:00.026708 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:16:00.026708 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:16:00.026708 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:16:00.026708 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:16:00.026708 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 15 19:16:00.026708 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:16:00.026708 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 488/966] 10.222.0.33 () {56 vars in 1120 bytes} [Fri Jul 15 19:15:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:16:00.031946 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a331f1cc-376e-43d8-827c-b2695bd4b00a None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:00.047415 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dbb778-0464-4c53-9dc0-bc54fbc6a414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:00.048277 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dbb778-0464-4c53-9dc0-bc54fbc6a414 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:00.048564 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dbb778-0464-4c53-9dc0-bc54fbc6a414 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:00.048889 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dbb778-0464-4c53-9dc0-bc54fbc6a414 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:00.057948 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dbb778-0464-4c53-9dc0-bc54fbc6a414 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:00.059658 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dbb778-0464-4c53-9dc0-bc54fbc6a414 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:00.075233 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-42dbb778-0464-4c53-9dc0-bc54fbc6a414 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 15 19:16:00.075233 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:16:00.075233 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:16:00.075233 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:16:00.075233 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:16:00.075233 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:16:00.075233 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:16:00.075233 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:16:00.075233 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:16:00.075233 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:16:00.075233 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:16:00.075233 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:16:00.075233 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 19:16:00.075233 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 19:16:00.075233 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 19:16:00.075233 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 19:16:00.076200 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:16:00.076200 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:16:00.076200 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:16:00.076200 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:16:00.076200 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:16:00.076200 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:16:00.076200 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:16:00.076200 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:16:00.076200 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:16:00.076200 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:16:00.076200 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:16:00.076200 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:16:00.076200 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:16:00.076200 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:16:00.076200 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:16:00.076200 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:16:00.076200 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:16:00.076200 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:16:00.077163 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:16:00.077163 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:16:00.077163 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 19:16:00.077163 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 19:16:00.077163 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 19:16:00.077163 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 19:16:00.077163 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 19:16:00.077163 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 19:16:00.077163 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 15 19:16:00.077163 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:16:00.080733 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 479/967] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 15 19:15:59 2022] GET /identity/v3/users/cinder => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:16:00.082069 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dbb778-0464-4c53-9dc0-bc54fbc6a414 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:00.148235 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a331f1cc-376e-43d8-827c-b2695bd4b00a None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:00.187502 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dbb778-0464-4c53-9dc0-bc54fbc6a414 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:00.194234 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ad01c0-08d6-45a9-a02d-a3a5f0e8e79b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:00.195191 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ad01c0-08d6-45a9-a02d-a3a5f0e8e79b None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:00.195674 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ad01c0-08d6-45a9-a02d-a3a5f0e8e79b None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:00.196170 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ad01c0-08d6-45a9-a02d-a3a5f0e8e79b None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:00.197115 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ad01c0-08d6-45a9-a02d-a3a5f0e8e79b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:16:00.197641 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ad01c0-08d6-45a9-a02d-a3a5f0e8e79b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:00.199222 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ad01c0-08d6-45a9-a02d-a3a5f0e8e79b None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:00.209497 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 489/968] 10.222.0.33 () {56 vars in 1115 bytes} [Fri Jul 15 19:16:00 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:16:00.215101 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ad01c0-08d6-45a9-a02d-a3a5f0e8e79b None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:00.224796 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:00.229474 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:00.231276 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4526a45-e01c-44e5-baf3-f836655241e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:00.232088 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4526a45-e01c-44e5-baf3-f836655241e3 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:00.232435 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4526a45-e01c-44e5-baf3-f836655241e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:00.232809 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4526a45-e01c-44e5-baf3-f836655241e3 None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:00.233727 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4526a45-e01c-44e5-baf3-f836655241e3 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:16:00.234160 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4526a45-e01c-44e5-baf3-f836655241e3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:00.235689 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4526a45-e01c-44e5-baf3-f836655241e3 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:00.282004 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 480/969] 10.222.0.33 () {56 vars in 1101 bytes} [Fri Jul 15 19:16:00 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:16:00.333647 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ad01c0-08d6-45a9-a02d-a3a5f0e8e79b None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:00.371462 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b01db-3608-4ccb-8f94-d84765feb121 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:00.371756 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b01db-3608-4ccb-8f94-d84765feb121 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:00.372040 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b01db-3608-4ccb-8f94-d84765feb121 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:00.374275 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 481/970] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:16:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 19:16:00.378998 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b1e4bf-ce88-46c9-a703-bf86d0233c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:00.380069 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391db164-1d4e-4960-af47-f13f84a20faf None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:00.380301 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b1e4bf-ce88-46c9-a703-bf86d0233c10 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:00.380424 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391db164-1d4e-4960-af47-f13f84a20faf None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:00.380636 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391db164-1d4e-4960-af47-f13f84a20faf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:00.381023 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b1e4bf-ce88-46c9-a703-bf86d0233c10 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:00.381449 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b1e4bf-ce88-46c9-a703-bf86d0233c10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:00.395966 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b1e4bf-ce88-46c9-a703-bf86d0233c10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:00.397685 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b1e4bf-ce88-46c9-a703-bf86d0233c10 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:00.400131 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-391db164-1d4e-4960-af47-f13f84a20faf None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:16:00.401807 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-391db164-1d4e-4960-af47-f13f84a20faf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:16:00.419070 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-e1b1e4bf-ce88-46c9-a703-bf86d0233c10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:16:00.419070 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:16:00.419070 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:16:00.419070 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:16:00.419070 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:16:00.419070 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:16:00.419070 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:16:00.419070 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:16:00.419070 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:16:00.419070 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:16:00.419070 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:16:00.419070 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:16:00.419070 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:16:00.419070 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:16:00.419070 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:16:00.420198 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:16:00.420198 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:16:00.420198 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:16:00.420198 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:16:00.420198 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:16:00.420198 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:16:00.420198 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:16:00.420198 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:16:00.420198 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:16:00.420198 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:16:00.420198 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:16:00.420198 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:16:00.420198 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:16:00.420198 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:16:00.420198 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:16:00.420198 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:16:00.420198 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:16:00.421316 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:16:00.421316 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:16:00.421316 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:16:00.421316 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:16:00.427840 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 490/971] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:16:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:16:00.428979 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b1e4bf-ce88-46c9-a703-bf86d0233c10 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:00.483326 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 482/972] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:16:00 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:16:00.497744 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:00.507435 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391db164-1d4e-4960-af47-f13f84a20faf None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:00.550249 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b1e4bf-ce88-46c9-a703-bf86d0233c10 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:00.601713 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca42906-3cd5-487c-904a-fe3ada80bcde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:00.602406 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca42906-3cd5-487c-904a-fe3ada80bcde None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:00.605849 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca42906-3cd5-487c-904a-fe3ada80bcde None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:00.605849 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca42906-3cd5-487c-904a-fe3ada80bcde None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:00.605849 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca42906-3cd5-487c-904a-fe3ada80bcde None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:16:00.605849 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca42906-3cd5-487c-904a-fe3ada80bcde None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:00.605849 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca42906-3cd5-487c-904a-fe3ada80bcde None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:00.618352 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 491/973] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:16:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:16:00.623804 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca42906-3cd5-487c-904a-fe3ada80bcde None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:00.641361 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391db164-1d4e-4960-af47-f13f84a20faf None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:00.661607 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:00.682471 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb55269-3d0d-42c8-93be-b1f23b678966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:00.683555 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb55269-3d0d-42c8-93be-b1f23b678966 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:00.683899 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb55269-3d0d-42c8-93be-b1f23b678966 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:00.684247 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb55269-3d0d-42c8-93be-b1f23b678966 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:00.684969 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb55269-3d0d-42c8-93be-b1f23b678966 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:00.686461 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb55269-3d0d-42c8-93be-b1f23b678966 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:00.699707 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-7bb55269-3d0d-42c8-93be-b1f23b678966 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 19:16:00.699707 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:16:00.699707 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 19:16:00.699707 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 19:16:00.699707 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 19:16:00.699707 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 19:16:00.699707 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 19:16:00.699707 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:16:00.699707 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 19:16:00.699707 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:16:00.699707 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 19:16:00.699707 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:16:00.699707 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:16:00.699707 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:16:00.699707 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:16:00.699707 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:16:00.699707 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:16:00.700972 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:16:00.700972 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:16:00.700972 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:16:00.700972 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:16:00.700972 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:16:00.700972 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:16:00.700972 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 19:16:00.700972 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 19:16:00.700972 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 19:16:00.700972 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 19:16:00.700972 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:16:00.700972 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:16:00.700972 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:16:00.700972 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:16:00.700972 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:16:00.700972 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:16:00.702022 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:16:00.702022 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:16:00.702022 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:16:00.702022 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:16:00.702022 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:16:00.702022 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:16:00.702022 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:16:00.702022 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:16:00.702022 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:16:00.702022 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:16:00.702022 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 19:16:00.702022 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:16:00.702022 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 19:16:00.702022 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:16:00.705107 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 483/974] 10.222.0.33 () {56 vars in 1126 bytes} [Fri Jul 15 19:16:00 2022] GET /identity/v3/services/object-store => generated 93 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:16:00.706689 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb55269-3d0d-42c8-93be-b1f23b678966 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:00.733649 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca42906-3cd5-487c-904a-fe3ada80bcde None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:00.756283 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:00.776617 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7bf895-da7f-4ac8-b66a-d08314dea6d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:00.778334 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7bf895-da7f-4ac8-b66a-d08314dea6d1 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:00.778723 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7bf895-da7f-4ac8-b66a-d08314dea6d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:00.779222 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7bf895-da7f-4ac8-b66a-d08314dea6d1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:00.787665 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7bf895-da7f-4ac8-b66a-d08314dea6d1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:00.789298 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7bf895-da7f-4ac8-b66a-d08314dea6d1 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:00.805502 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-2b7bf895-da7f-4ac8-b66a-d08314dea6d1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 15 19:16:00.805502 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:16:00.805502 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:16:00.805502 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:16:00.805502 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:16:00.805502 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:16:00.805502 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:16:00.805502 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:16:00.805502 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:16:00.805502 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:16:00.805502 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:16:00.805502 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:16:00.805502 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 19:16:00.805502 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 19:16:00.805502 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 19:16:00.805502 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 19:16:00.806736 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:16:00.806736 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:16:00.806736 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:16:00.806736 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:16:00.806736 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:16:00.806736 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:16:00.806736 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:16:00.806736 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:16:00.806736 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:16:00.806736 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:16:00.806736 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:16:00.806736 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:16:00.806736 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:16:00.806736 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:16:00.806736 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:16:00.806736 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:16:00.806736 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:16:00.806736 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:16:00.807915 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:16:00.807915 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:16:00.807915 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 19:16:00.807915 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 19:16:00.807915 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 19:16:00.807915 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 19:16:00.807915 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 19:16:00.807915 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 19:16:00.807915 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 15 19:16:00.807915 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:16:00.809297 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 492/975] 10.222.0.33 () {56 vars in 1117 bytes} [Fri Jul 15 19:16:00 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:16:00.814835 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7bf895-da7f-4ac8-b66a-d08314dea6d1 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:00.838761 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb55269-3d0d-42c8-93be-b1f23b678966 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:00.875809 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcdefaa-9578-4ce0-b29c-9d5a43b925df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:00.876475 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcdefaa-9578-4ce0-b29c-9d5a43b925df None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:00.876758 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcdefaa-9578-4ce0-b29c-9d5a43b925df None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:00.877035 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcdefaa-9578-4ce0-b29c-9d5a43b925df None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:00.877817 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcdefaa-9578-4ce0-b29c-9d5a43b925df None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:16:00.878133 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcdefaa-9578-4ce0-b29c-9d5a43b925df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:00.879728 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcdefaa-9578-4ce0-b29c-9d5a43b925df None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:00.889460 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 484/976] 10.222.0.33 () {56 vars in 1122 bytes} [Fri Jul 15 19:16:00 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:16:00.891491 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcdefaa-9578-4ce0-b29c-9d5a43b925df None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:00.923003 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7bf895-da7f-4ac8-b66a-d08314dea6d1 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:00.944351 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:00.960636 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ea2d17-23a9-4ad3-8695-de7c3e28a8f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:00.961311 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ea2d17-23a9-4ad3-8695-de7c3e28a8f6 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:00.961594 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ea2d17-23a9-4ad3-8695-de7c3e28a8f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:00.961904 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ea2d17-23a9-4ad3-8695-de7c3e28a8f6 None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:00.962755 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ea2d17-23a9-4ad3-8695-de7c3e28a8f6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:16:00.963067 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ea2d17-23a9-4ad3-8695-de7c3e28a8f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:00.964501 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ea2d17-23a9-4ad3-8695-de7c3e28a8f6 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:00.982299 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 493/977] 10.222.0.33 () {56 vars in 1149 bytes} [Fri Jul 15 19:16:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:16:00.987814 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ea2d17-23a9-4ad3-8695-de7c3e28a8f6 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:00.999281 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcdefaa-9578-4ce0-b29c-9d5a43b925df None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:01.032101 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:01.035901 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb0e440-e364-4b56-b9cb-1f5b7dd52208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:01.036904 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb0e440-e364-4b56-b9cb-1f5b7dd52208 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:01.037191 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb0e440-e364-4b56-b9cb-1f5b7dd52208 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:01.037479 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb0e440-e364-4b56-b9cb-1f5b7dd52208 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:01.038192 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb0e440-e364-4b56-b9cb-1f5b7dd52208 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:16:01.038493 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb0e440-e364-4b56-b9cb-1f5b7dd52208 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:01.039968 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb0e440-e364-4b56-b9cb-1f5b7dd52208 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:01.049948 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 485/978] 10.222.0.33 () {56 vars in 1122 bytes} [Fri Jul 15 19:16:00 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:16:01.051225 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb0e440-e364-4b56-b9cb-1f5b7dd52208 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:01.095221 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ea2d17-23a9-4ad3-8695-de7c3e28a8f6 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:01.132770 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f28d1e-92ad-4602-83b0-4c184e7cf030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:01.133576 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f28d1e-92ad-4602-83b0-4c184e7cf030 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:01.133908 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f28d1e-92ad-4602-83b0-4c184e7cf030 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:01.134189 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f28d1e-92ad-4602-83b0-4c184e7cf030 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:01.156436 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb0e440-e364-4b56-b9cb-1f5b7dd52208 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:01.168232 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:01.192709 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73bd795-c63c-482e-86c2-eab919acc111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:01.193377 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73bd795-c63c-482e-86c2-eab919acc111 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:01.193662 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73bd795-c63c-482e-86c2-eab919acc111 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:01.193966 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73bd795-c63c-482e-86c2-eab919acc111 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:01.194775 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73bd795-c63c-482e-86c2-eab919acc111 None admin] RBAC: Adding query filter params (interface=admin, service_id=e8870c0588ae49418f8532a6f8ec5491, region_id=RegionOne) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:16:01.195125 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73bd795-c63c-482e-86c2-eab919acc111 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:01.196510 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73bd795-c63c-482e-86c2-eab919acc111 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:01.203155 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 486/979] 10.222.0.33 () {56 vars in 1249 bytes} [Fri Jul 15 19:16:01 2022] GET /identity/v3/endpoints?service_id=e8870c0588ae49418f8532a6f8ec5491&interface=admin®ion_id=RegionOne => generated 193 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:16:01.208794 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f28d1e-92ad-4602-83b0-4c184e7cf030 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:01.210291 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f28d1e-92ad-4602-83b0-4c184e7cf030 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:01.210831 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73bd795-c63c-482e-86c2-eab919acc111 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:01.258129 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:01.321593 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 494/980] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 15 19:16:00 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 15 19:16:01.327357 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f28d1e-92ad-4602-83b0-4c184e7cf030 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:01.331798 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73bd795-c63c-482e-86c2-eab919acc111 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:01.367272 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99f661e-8ef2-42c4-b2cf-dc115f875a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:01.367861 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f661e-8ef2-42c4-b2cf-dc115f875a37 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:01.368144 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f661e-8ef2-42c4-b2cf-dc115f875a37 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:01.368421 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f661e-8ef2-42c4-b2cf-dc115f875a37 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:01.369097 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99f661e-8ef2-42c4-b2cf-dc115f875a37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:01.370498 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99f661e-8ef2-42c4-b2cf-dc115f875a37 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:01.376622 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 487/981] 10.222.0.33 () {56 vars in 1087 bytes} [Fri Jul 15 19:16:01 2022] GET /identity/v3/services => generated 1497 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:16:01.377835 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85300e2b-f4f7-45c5-b499-755723058fc6 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:01.378125 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85300e2b-f4f7-45c5-b499-755723058fc6 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:01.378402 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85300e2b-f4f7-45c5-b499-755723058fc6 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:01.379642 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 488/982] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:16:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:16:01.385445 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af69d333-e957-4d3c-a78c-6ddc0bfb9597 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:01.385885 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af69d333-e957-4d3c-a78c-6ddc0bfb9597 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:01.386170 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af69d333-e957-4d3c-a78c-6ddc0bfb9597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:01.403890 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-af69d333-e957-4d3c-a78c-6ddc0bfb9597 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:16:01.405137 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-af69d333-e957-4d3c-a78c-6ddc0bfb9597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:16:01.444852 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:01.445663 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f28d1e-92ad-4602-83b0-4c184e7cf030 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:01.483026 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a406689b-01a4-41ec-9ac0-cc24af9c7f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:01.483836 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a406689b-01a4-41ec-9ac0-cc24af9c7f9f None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:01.484120 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a406689b-01a4-41ec-9ac0-cc24af9c7f9f None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:01.484395 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a406689b-01a4-41ec-9ac0-cc24af9c7f9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:01.492678 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 489/983] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:16:01 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 19:16:01.493303 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:01.501951 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a406689b-01a4-41ec-9ac0-cc24af9c7f9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:01.503558 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a406689b-01a4-41ec-9ac0-cc24af9c7f9f None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:01.516430 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af69d333-e957-4d3c-a78c-6ddc0bfb9597 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:01.525665 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-a406689b-01a4-41ec-9ac0-cc24af9c7f9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:16:01.525665 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:16:01.525665 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:16:01.525665 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:16:01.525665 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:16:01.525665 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:16:01.525665 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:16:01.525665 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:16:01.525665 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:16:01.525665 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:16:01.525665 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:16:01.525665 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:16:01.525665 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:16:01.525665 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:16:01.525665 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:16:01.526685 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:16:01.526685 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:16:01.526685 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:16:01.526685 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:16:01.526685 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:16:01.526685 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:16:01.526685 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:16:01.526685 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:16:01.526685 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:16:01.526685 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:16:01.526685 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:16:01.526685 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:16:01.526685 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:16:01.526685 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:16:01.526685 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:16:01.526685 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:16:01.526685 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:16:01.527859 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:16:01.527859 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:16:01.527859 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:16:01.527859 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:16:01.527859 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 495/984] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:16:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:16:01.534360 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a406689b-01a4-41ec-9ac0-cc24af9c7f9f None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:01.628163 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af69d333-e957-4d3c-a78c-6ddc0bfb9597 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:01.642703 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a406689b-01a4-41ec-9ac0-cc24af9c7f9f None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:01.664806 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47605b65-d45a-44a1-b6c8-3187217289da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:01.665473 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47605b65-d45a-44a1-b6c8-3187217289da None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:01.665755 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47605b65-d45a-44a1-b6c8-3187217289da None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:01.666050 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47605b65-d45a-44a1-b6c8-3187217289da None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:01.666774 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47605b65-d45a-44a1-b6c8-3187217289da None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:01.668142 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47605b65-d45a-44a1-b6c8-3187217289da None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:01.680026 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94e017b-a858-4da2-9840-0edfb351e3fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:01.680691 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94e017b-a858-4da2-9840-0edfb351e3fa None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:01.680977 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94e017b-a858-4da2-9840-0edfb351e3fa None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:01.681275 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94e017b-a858-4da2-9840-0edfb351e3fa None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:01.682005 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94e017b-a858-4da2-9840-0edfb351e3fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:16:01.690257 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94e017b-a858-4da2-9840-0edfb351e3fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:01.690257 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94e017b-a858-4da2-9840-0edfb351e3fa None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:01.695147 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 496/985] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:16:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:16:01.700764 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94e017b-a858-4da2-9840-0edfb351e3fa None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:01.745944 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 490/986] 10.222.0.33 () {60 vars in 1141 bytes} [Fri Jul 15 19:16:01 2022] POST /identity/v3/services => generated 242 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:16:01.764133 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:01.828302 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94e017b-a858-4da2-9840-0edfb351e3fa None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:01.873378 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae71c167-5426-476c-8376-6beeb09611be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:01.874224 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae71c167-5426-476c-8376-6beeb09611be None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:01.874504 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae71c167-5426-476c-8376-6beeb09611be None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:01.874775 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae71c167-5426-476c-8376-6beeb09611be None admin] PATH_INFO: `/v3/projects/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:01.883903 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae71c167-5426-476c-8376-6beeb09611be None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:01.885282 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae71c167-5426-476c-8376-6beeb09611be None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:01.901562 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-ae71c167-5426-476c-8376-6beeb09611be None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:16:01.901562 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:16:01.901562 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:16:01.901562 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:16:01.901562 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:16:01.901562 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:16:01.901562 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:16:01.901562 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:16:01.901562 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:16:01.901562 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:16:01.901562 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:16:01.901562 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:16:01.901562 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 19:16:01.901562 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 19:16:01.901562 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 19:16:01.902836 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 19:16:01.902836 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:16:01.902836 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:16:01.902836 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:16:01.902836 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:16:01.902836 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:16:01.902836 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:16:01.902836 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:16:01.902836 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:16:01.902836 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:16:01.902836 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:16:01.902836 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:16:01.902836 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:16:01.902836 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:16:01.902836 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:16:01.902836 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:16:01.902836 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:16:01.905015 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 19:16:01.905015 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 19:16:01.905015 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:16:01.905015 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:16:01.905015 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:16:01.905015 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:16:01.905015 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:16:01.905015 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:16:01.905015 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 497/987] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 15 19:16:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:16:01.909825 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47605b65-d45a-44a1-b6c8-3187217289da None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:01.942980 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:02.017653 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47605b65-d45a-44a1-b6c8-3187217289da None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:02.053543 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dc50b9-71ed-4208-97dd-a2f83741d8d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:02.054246 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dc50b9-71ed-4208-97dd-a2f83741d8d6 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:02.054530 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dc50b9-71ed-4208-97dd-a2f83741d8d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:02.054805 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dc50b9-71ed-4208-97dd-a2f83741d8d6 None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:02.055590 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dc50b9-71ed-4208-97dd-a2f83741d8d6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:16:02.055898 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dc50b9-71ed-4208-97dd-a2f83741d8d6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:02.057335 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dc50b9-71ed-4208-97dd-a2f83741d8d6 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:02.075906 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 491/988] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jul 15 19:16:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:16:02.077553 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae71c167-5426-476c-8376-6beeb09611be None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:02.179140 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae71c167-5426-476c-8376-6beeb09611be None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:02.186410 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:02.220426 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1c8b27-601f-495f-a2c1-c7b42f614281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:02.221109 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c8b27-601f-495f-a2c1-c7b42f614281 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:02.221393 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c8b27-601f-495f-a2c1-c7b42f614281 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:02.221673 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c8b27-601f-495f-a2c1-c7b42f614281 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:02.222591 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1c8b27-601f-495f-a2c1-c7b42f614281 None admin] RBAC: Adding query filter params (role.id=c1dc7198be2d43d3aa44d16107d9a959, scope.project.id=55a756ccaef14cb49b26b0d68dd0b466, user.id=3d893c14620b4a129842062f14a68a78) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:16:02.222906 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1c8b27-601f-495f-a2c1-c7b42f614281 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:02.224345 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1c8b27-601f-495f-a2c1-c7b42f614281 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:02.237803 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 498/989] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 15 19:16:02 2022] GET /identity/v3/role_assignments?user.id=3d893c14620b4a129842062f14a68a78&scope.project.id=55a756ccaef14cb49b26b0d68dd0b466&role.id=c1dc7198be2d43d3aa44d16107d9a959 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:16:03.398481 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc8f5b-fa83-4112-8d6d-af3e8131afe7 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:03.399010 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc8f5b-fa83-4112-8d6d-af3e8131afe7 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:03.399010 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc8f5b-fa83-4112-8d6d-af3e8131afe7 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:03.400535 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 492/990] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:16:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:16:03.405510 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c550719-acc1-4fcb-8c39-75092d674f97 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:03.405934 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c550719-acc1-4fcb-8c39-75092d674f97 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:03.406134 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c550719-acc1-4fcb-8c39-75092d674f97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:03.422904 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-0c550719-acc1-4fcb-8c39-75092d674f97 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:16:03.424102 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-0c550719-acc1-4fcb-8c39-75092d674f97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:16:03.490908 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 499/991] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:16:03 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:16:03.512954 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dc8f5b-fa83-4112-8d6d-af3e8131afe7 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:03.602597 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:03.616812 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dc8f5b-fa83-4112-8d6d-af3e8131afe7 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:03.652328 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6076fc-eb7c-4308-82c7-2e4e24782702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:03.653062 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6076fc-eb7c-4308-82c7-2e4e24782702 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:03.653345 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6076fc-eb7c-4308-82c7-2e4e24782702 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:03.653621 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6076fc-eb7c-4308-82c7-2e4e24782702 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:03.654306 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6076fc-eb7c-4308-82c7-2e4e24782702 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:03.655676 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6076fc-eb7c-4308-82c7-2e4e24782702 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:03.668509 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-1d6076fc-eb7c-4308-82c7-2e4e24782702 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 19:16:03.668509 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:16:03.668509 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 19:16:03.668509 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 19:16:03.668509 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 19:16:03.668509 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 19:16:03.668509 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 19:16:03.668509 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:16:03.668509 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 19:16:03.668509 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:16:03.668509 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 19:16:03.668509 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:16:03.668509 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:16:03.668509 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:16:03.668509 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:16:03.668509 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:16:03.668509 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:16:03.669574 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:16:03.669574 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:16:03.669574 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:16:03.669574 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:16:03.669574 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:16:03.669574 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:16:03.669574 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 19:16:03.669574 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 19:16:03.669574 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 19:16:03.669574 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 19:16:03.669574 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:16:03.669574 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:16:03.669574 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:16:03.669574 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:16:03.669574 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:16:03.669574 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:16:03.670558 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:16:03.670558 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:16:03.670558 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:16:03.670558 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:16:03.670558 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:16:03.670558 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:16:03.670558 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:16:03.670558 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:16:03.670558 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:16:03.670558 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:16:03.670558 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 19:16:03.670558 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:16:03.670558 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 19:16:03.670558 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:16:03.670558 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 493/992] 10.222.0.33 () {56 vars in 1126 bytes} [Fri Jul 15 19:16:03 2022] GET /identity/v3/services/object-store => generated 93 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:16:03.677433 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c550719-acc1-4fcb-8c39-75092d674f97 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:03.781364 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:03.785036 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c550719-acc1-4fcb-8c39-75092d674f97 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:03.802961 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61def2bc-d59e-472a-97c8-e1d12a91e974 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:03.803241 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61def2bc-d59e-472a-97c8-e1d12a91e974 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:03.803521 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61def2bc-d59e-472a-97c8-e1d12a91e974 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:03.805056 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 494/993] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:16:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 19:16:03.810236 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d650d955-64f8-4b57-ba1d-7f4d72671d65 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:03.810529 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d650d955-64f8-4b57-ba1d-7f4d72671d65 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:03.810810 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d650d955-64f8-4b57-ba1d-7f4d72671d65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:03.824860 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdb958f-87fc-48de-9be7-aaea962bcb48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:03.825791 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdb958f-87fc-48de-9be7-aaea962bcb48 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:03.826216 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdb958f-87fc-48de-9be7-aaea962bcb48 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:03.826649 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdb958f-87fc-48de-9be7-aaea962bcb48 None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:03.827491 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdb958f-87fc-48de-9be7-aaea962bcb48 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:16:03.827744 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-d650d955-64f8-4b57-ba1d-7f4d72671d65 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:16:03.828101 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdb958f-87fc-48de-9be7-aaea962bcb48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:03.828936 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d650d955-64f8-4b57-ba1d-7f4d72671d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:16:03.829758 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdb958f-87fc-48de-9be7-aaea962bcb48 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:03.836360 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 500/994] 10.222.0.33 () {56 vars in 1122 bytes} [Fri Jul 15 19:16:03 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:16:03.842218 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdb958f-87fc-48de-9be7-aaea962bcb48 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:03.917910 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 495/995] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:16:03 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 19:16:03.940709 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d650d955-64f8-4b57-ba1d-7f4d72671d65 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:03.968034 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdb958f-87fc-48de-9be7-aaea962bcb48 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:04.006984 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10605c75-d2ef-42ab-a007-3b4cc94e8c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:04.008222 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10605c75-d2ef-42ab-a007-3b4cc94e8c0b None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:04.008518 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10605c75-d2ef-42ab-a007-3b4cc94e8c0b None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:04.008798 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10605c75-d2ef-42ab-a007-3b4cc94e8c0b None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:04.009513 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10605c75-d2ef-42ab-a007-3b4cc94e8c0b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:16:04.009842 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10605c75-d2ef-42ab-a007-3b4cc94e8c0b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:04.011941 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10605c75-d2ef-42ab-a007-3b4cc94e8c0b None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:04.023907 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 501/996] 10.222.0.33 () {56 vars in 1122 bytes} [Fri Jul 15 19:16:03 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:16:04.024535 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10605c75-d2ef-42ab-a007-3b4cc94e8c0b None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:04.029182 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:04.051002 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d650d955-64f8-4b57-ba1d-7f4d72671d65 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:04.053892 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:04.089363 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c849757c-498a-4bed-a35d-3fbe10032f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:04.090194 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c849757c-498a-4bed-a35d-3fbe10032f61 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:04.090476 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c849757c-498a-4bed-a35d-3fbe10032f61 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:04.090750 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c849757c-498a-4bed-a35d-3fbe10032f61 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:04.091473 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c849757c-498a-4bed-a35d-3fbe10032f61 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:04.093337 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c849757c-498a-4bed-a35d-3fbe10032f61 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:04.105258 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-c849757c-498a-4bed-a35d-3fbe10032f61 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 15 19:16:04.105258 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:16:04.105258 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 19:16:04.105258 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 19:16:04.105258 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 19:16:04.105258 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 19:16:04.105258 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 19:16:04.105258 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:16:04.105258 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 15 19:16:04.105258 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:16:04.105258 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 19:16:04.105258 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:16:04.105258 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:16:04.105258 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:16:04.105258 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:16:04.105258 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:16:04.105258 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:16:04.106320 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:16:04.106320 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:16:04.106320 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:16:04.106320 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:16:04.106320 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:16:04.106320 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:16:04.106320 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 19:16:04.106320 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 19:16:04.106320 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 19:16:04.106320 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 19:16:04.106320 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:16:04.106320 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:16:04.106320 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:16:04.106320 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:16:04.106320 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:16:04.106320 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:16:04.107192 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:16:04.107192 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:16:04.107192 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:16:04.107192 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:16:04.107192 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:16:04.107192 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:16:04.107192 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:16:04.107192 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:16:04.107192 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:16:04.107192 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:16:04.107192 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 19:16:04.107192 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:16:04.107192 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 15 19:16:04.107192 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:16:04.107192 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 496/997] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 15 19:16:03 2022] GET /identity/v3/services/compute => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:16:04.112198 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c849757c-498a-4bed-a35d-3fbe10032f61 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:04.135574 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10605c75-d2ef-42ab-a007-3b4cc94e8c0b None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:04.173893 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751dc24e-0039-4ae5-8d38-a4e3e4bdf293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:04.174457 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751dc24e-0039-4ae5-8d38-a4e3e4bdf293 None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:04.174738 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751dc24e-0039-4ae5-8d38-a4e3e4bdf293 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:04.175508 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751dc24e-0039-4ae5-8d38-a4e3e4bdf293 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:04.176264 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751dc24e-0039-4ae5-8d38-a4e3e4bdf293 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:04.177623 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751dc24e-0039-4ae5-8d38-a4e3e4bdf293 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:04.228230 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c849757c-498a-4bed-a35d-3fbe10032f61 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:04.256222 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 502/998] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 15 19:16:04 2022] POST /identity/v3/endpoints => generated 325 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:16:04.278976 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a7a56c-b49a-48c6-a7f4-3bff0a7e2541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:04.279670 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7a56c-b49a-48c6-a7f4-3bff0a7e2541 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:04.279953 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7a56c-b49a-48c6-a7f4-3bff0a7e2541 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:04.280230 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7a56c-b49a-48c6-a7f4-3bff0a7e2541 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:04.280955 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a7a56c-b49a-48c6-a7f4-3bff0a7e2541 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:16:04.281290 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a7a56c-b49a-48c6-a7f4-3bff0a7e2541 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:04.282764 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a7a56c-b49a-48c6-a7f4-3bff0a7e2541 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:04.293546 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 497/999] 10.222.0.33 () {56 vars in 1112 bytes} [Fri Jul 15 19:16:04 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:16:04.294960 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751dc24e-0039-4ae5-8d38-a4e3e4bdf293 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:04.322798 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bea83b1-f723-4f5f-ba6e-b2d65775d4f9 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:04.323148 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bea83b1-f723-4f5f-ba6e-b2d65775d4f9 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:04.323541 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bea83b1-f723-4f5f-ba6e-b2d65775d4f9 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:04.325236 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 498/1000] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:16:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:16:04.331990 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630a0510-9035-4516-b28a-a2032c2d66b5 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:04.332284 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630a0510-9035-4516-b28a-a2032c2d66b5 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:04.334204 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630a0510-9035-4516-b28a-a2032c2d66b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:04.345621 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:04.352193 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-630a0510-9035-4516-b28a-a2032c2d66b5 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:16:04.353541 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-630a0510-9035-4516-b28a-a2032c2d66b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:16:04.397512 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:04.414126 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751dc24e-0039-4ae5-8d38-a4e3e4bdf293 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:04.436722 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 499/1001] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:16:04 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 19:16:04.453044 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630a0510-9035-4516-b28a-a2032c2d66b5 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:04.453686 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c483c5cf-4a38-49c7-ac31-b849cbb12a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:04.454408 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c483c5cf-4a38-49c7-ac31-b849cbb12a77 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:04.454694 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c483c5cf-4a38-49c7-ac31-b849cbb12a77 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:04.455139 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c483c5cf-4a38-49c7-ac31-b849cbb12a77 None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:04.455840 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c483c5cf-4a38-49c7-ac31-b849cbb12a77 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:16:04.456268 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c483c5cf-4a38-49c7-ac31-b849cbb12a77 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:04.457843 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c483c5cf-4a38-49c7-ac31-b849cbb12a77 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:04.463261 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 503/1002] 10.222.0.33 () {56 vars in 1112 bytes} [Fri Jul 15 19:16:04 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:16:04.469638 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c483c5cf-4a38-49c7-ac31-b849cbb12a77 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:04.559217 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630a0510-9035-4516-b28a-a2032c2d66b5 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:04.628137 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c483c5cf-4a38-49c7-ac31-b849cbb12a77 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:04.631163 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989c31c3-ce70-48fd-a511-d472361e0759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:04.631417 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989c31c3-ce70-48fd-a511-d472361e0759 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:04.631670 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989c31c3-ce70-48fd-a511-d472361e0759 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:04.632089 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989c31c3-ce70-48fd-a511-d472361e0759 None admin] PATH_INFO: `/v3/services/image` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:04.632854 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989c31c3-ce70-48fd-a511-d472361e0759 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:04.634288 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989c31c3-ce70-48fd-a511-d472361e0759 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:04.637674 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:04.648204 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-989c31c3-ce70-48fd-a511-d472361e0759 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 15 19:16:04.648204 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:16:04.648204 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 19:16:04.648204 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 19:16:04.648204 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 19:16:04.648204 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 19:16:04.648204 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 19:16:04.648204 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:16:04.648204 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 15 19:16:04.648204 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:16:04.648204 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 19:16:04.648204 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:16:04.648204 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:16:04.648204 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:16:04.648204 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:16:04.648204 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:16:04.648204 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:16:04.649346 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:16:04.649346 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:16:04.649346 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:16:04.649346 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:16:04.649346 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:16:04.649346 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:16:04.649346 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 19:16:04.649346 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 19:16:04.649346 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 19:16:04.649346 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 19:16:04.649346 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:16:04.649346 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:16:04.649346 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:16:04.649346 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:16:04.649346 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:16:04.649346 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:16:04.650288 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:16:04.650288 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:16:04.650288 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:16:04.650288 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:16:04.650288 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:16:04.650288 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:16:04.650288 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:16:04.650288 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:16:04.650288 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:16:04.650288 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:16:04.650288 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 19:16:04.650288 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:16:04.650288 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 15 19:16:04.650288 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:16:04.654565 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 500/1003] 10.222.0.33 () {56 vars in 1105 bytes} [Fri Jul 15 19:16:04 2022] GET /identity/v3/services/image => generated 86 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:16:04.655204 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989c31c3-ce70-48fd-a511-d472361e0759 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:04.667673 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d9e8ee-200a-4cb6-904e-4d5392a6f633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:04.668338 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d9e8ee-200a-4cb6-904e-4d5392a6f633 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:04.668621 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d9e8ee-200a-4cb6-904e-4d5392a6f633 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:04.668903 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d9e8ee-200a-4cb6-904e-4d5392a6f633 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:04.669726 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d9e8ee-200a-4cb6-904e-4d5392a6f633 None admin] RBAC: Adding query filter params (interface=public, service_id=e9837d60e4264ca4893129ade04cd827, region_id=RegionOne) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:16:04.670088 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d9e8ee-200a-4cb6-904e-4d5392a6f633 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:04.671444 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d9e8ee-200a-4cb6-904e-4d5392a6f633 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:04.681945 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 504/1004] 10.222.0.33 () {56 vars in 1251 bytes} [Fri Jul 15 19:16:04 2022] GET /identity/v3/endpoints?service_id=e9837d60e4264ca4893129ade04cd827&interface=public®ion_id=RegionOne => generated 194 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:16:04.683439 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d9e8ee-200a-4cb6-904e-4d5392a6f633 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:04.708747 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:04.766211 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989c31c3-ce70-48fd-a511-d472361e0759 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:04.794757 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d9e8ee-200a-4cb6-904e-4d5392a6f633 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:04.806127 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b20c24-1e89-4461-a33c-e3dd47ec11d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:04.806805 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b20c24-1e89-4461-a33c-e3dd47ec11d0 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:04.807106 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b20c24-1e89-4461-a33c-e3dd47ec11d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:04.807357 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b20c24-1e89-4461-a33c-e3dd47ec11d0 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:04.808072 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b20c24-1e89-4461-a33c-e3dd47ec11d0 None admin] RBAC: Adding query filter params (name=image) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:16:04.808377 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b20c24-1e89-4461-a33c-e3dd47ec11d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:04.809751 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b20c24-1e89-4461-a33c-e3dd47ec11d0 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:04.820207 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 501/1005] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:16:04 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:16:04.821265 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b20c24-1e89-4461-a33c-e3dd47ec11d0 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:04.841430 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cd9207-7519-442c-a934-3983517f61a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:04.842142 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cd9207-7519-442c-a934-3983517f61a5 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:04.842441 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cd9207-7519-442c-a934-3983517f61a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:04.842699 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cd9207-7519-442c-a934-3983517f61a5 None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:04.843375 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cd9207-7519-442c-a934-3983517f61a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:04.844764 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cd9207-7519-442c-a934-3983517f61a5 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:04.851411 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 505/1006] 10.222.0.33 () {56 vars in 1087 bytes} [Fri Jul 15 19:16:04 2022] GET /identity/v3/services => generated 1727 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 19:16:04.888616 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:04.933013 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b20c24-1e89-4461-a33c-e3dd47ec11d0 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:04.970012 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaa01da-16b5-4d37-a755-9b3b50a7324c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:04.970594 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaa01da-16b5-4d37-a755-9b3b50a7324c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:04.970881 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaa01da-16b5-4d37-a755-9b3b50a7324c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:04.971161 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaa01da-16b5-4d37-a755-9b3b50a7324c None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:04.972469 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaa01da-16b5-4d37-a755-9b3b50a7324c None admin] RBAC: Adding query filter params (type=image) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:16:04.972777 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaa01da-16b5-4d37-a755-9b3b50a7324c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:04.974210 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaa01da-16b5-4d37-a755-9b3b50a7324c None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:04.986074 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 502/1007] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:16:04 2022] GET /identity/v3/services?type=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:16:06.322107 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb9fb5a-1d9c-477e-97d7-d7d8f309b044 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:06.322618 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb9fb5a-1d9c-477e-97d7-d7d8f309b044 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:06.322697 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb9fb5a-1d9c-477e-97d7-d7d8f309b044 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:06.324216 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 506/1008] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:16:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:16:06.329164 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f94a66-f49b-4344-941d-a1dd41a068ca None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:06.329562 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f94a66-f49b-4344-941d-a1dd41a068ca None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:06.329718 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f94a66-f49b-4344-941d-a1dd41a068ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:06.346898 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-72f94a66-f49b-4344-941d-a1dd41a068ca None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:16:06.348086 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-72f94a66-f49b-4344-941d-a1dd41a068ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:16:06.417973 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 503/1009] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:16:06 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:16:06.441272 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb9fb5a-1d9c-477e-97d7-d7d8f309b044 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:06.524268 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:06.544785 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb9fb5a-1d9c-477e-97d7-d7d8f309b044 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:06.586553 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50e86b0-a811-4645-b2e4-7de55ee6cbc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:06.587259 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50e86b0-a811-4645-b2e4-7de55ee6cbc8 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:06.587544 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50e86b0-a811-4645-b2e4-7de55ee6cbc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:06.587826 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50e86b0-a811-4645-b2e4-7de55ee6cbc8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:06.589049 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50e86b0-a811-4645-b2e4-7de55ee6cbc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:06.590525 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50e86b0-a811-4645-b2e4-7de55ee6cbc8 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:06.598186 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 507/1010] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:16:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:16:06.599798 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f94a66-f49b-4344-941d-a1dd41a068ca None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:06.659826 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:06.701686 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f94a66-f49b-4344-941d-a1dd41a068ca None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:06.735903 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd3e3fc-0289-4da6-a6f0-5622362bac02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:06.736606 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd3e3fc-0289-4da6-a6f0-5622362bac02 None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:06.736859 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd3e3fc-0289-4da6-a6f0-5622362bac02 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:06.737134 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd3e3fc-0289-4da6-a6f0-5622362bac02 None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:06.737943 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd3e3fc-0289-4da6-a6f0-5622362bac02 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:06.739388 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd3e3fc-0289-4da6-a6f0-5622362bac02 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:06.844161 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 504/1011] 10.222.0.33 () {60 vars in 1141 bytes} [Fri Jul 15 19:16:06 2022] POST /identity/v3/projects => generated 311 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:16:06.871970 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7268c340-21fa-4faa-be36-e6825d55d2fd None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:06.872292 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7268c340-21fa-4faa-be36-e6825d55d2fd None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:06.872535 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7268c340-21fa-4faa-be36-e6825d55d2fd None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:06.874174 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 508/1012] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:16:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 19:16:06.880480 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19213c78-59df-4e75-ae16-2c9fa275bb96 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:06.880813 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19213c78-59df-4e75-ae16-2c9fa275bb96 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:06.881066 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19213c78-59df-4e75-ae16-2c9fa275bb96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:06.901212 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-19213c78-59df-4e75-ae16-2c9fa275bb96 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:16:06.902525 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-19213c78-59df-4e75-ae16-2c9fa275bb96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:16:06.978930 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 505/1013] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:16:06 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:16:07.002122 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7268c340-21fa-4faa-be36-e6825d55d2fd None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:07.011216 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e423de-b011-4550-a36d-77da4373df19 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:07.011509 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e423de-b011-4550-a36d-77da4373df19 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:07.011797 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e423de-b011-4550-a36d-77da4373df19 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:07.013326 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 506/1014] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:16:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:16:07.018075 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c7886d-057b-434a-bcd1-a2c16cee06e9 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:07.018370 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c7886d-057b-434a-bcd1-a2c16cee06e9 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:07.018696 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c7886d-057b-434a-bcd1-a2c16cee06e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:07.037042 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-82c7886d-057b-434a-bcd1-a2c16cee06e9 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:16:07.038254 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-82c7886d-057b-434a-bcd1-a2c16cee06e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:16:07.077162 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:07.113228 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 507/1015] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:16:07 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 19:16:07.117267 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7268c340-21fa-4faa-be36-e6825d55d2fd None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:07.135801 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c7886d-057b-434a-bcd1-a2c16cee06e9 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:07.156191 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b7d2ce-32df-495e-9cfa-7d9be7ff2b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:07.156992 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b7d2ce-32df-495e-9cfa-7d9be7ff2b0d None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:07.157276 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b7d2ce-32df-495e-9cfa-7d9be7ff2b0d None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:07.157554 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b7d2ce-32df-495e-9cfa-7d9be7ff2b0d None admin] PATH_INFO: `/v3/services/compute` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:07.158246 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b7d2ce-32df-495e-9cfa-7d9be7ff2b0d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:07.160190 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b7d2ce-32df-495e-9cfa-7d9be7ff2b0d None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:07.172035 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-89b7d2ce-32df-495e-9cfa-7d9be7ff2b0d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 15 19:16:07.172035 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:16:07.172035 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 19:16:07.172035 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 19:16:07.172035 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 19:16:07.172035 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 19:16:07.172035 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 19:16:07.172035 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:16:07.172035 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 15 19:16:07.172035 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:16:07.172035 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 19:16:07.172035 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:16:07.172035 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:16:07.172035 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:16:07.172035 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:16:07.172035 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:16:07.172035 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:16:07.173063 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:16:07.173063 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:16:07.173063 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:16:07.173063 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:16:07.173063 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:16:07.173063 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:16:07.173063 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 19:16:07.173063 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 19:16:07.173063 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 19:16:07.173063 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 19:16:07.173063 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:16:07.173063 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:16:07.173063 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:16:07.173063 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:16:07.173063 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:16:07.173063 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:16:07.173984 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:16:07.173984 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:16:07.173984 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:16:07.173984 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:16:07.173984 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:16:07.173984 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:16:07.173984 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:16:07.173984 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:16:07.173984 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:16:07.173984 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:16:07.173984 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 19:16:07.173984 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:16:07.173984 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 15 19:16:07.173984 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:16:07.178043 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 509/1016] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 15 19:16:07 2022] GET /identity/v3/services/compute => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:16:07.179510 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b7d2ce-32df-495e-9cfa-7d9be7ff2b0d None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:07.199457 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:07.256956 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c7886d-057b-434a-bcd1-a2c16cee06e9 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:07.292496 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b7d2ce-32df-495e-9cfa-7d9be7ff2b0d None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:07.299774 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671dddd6-54f9-4edf-80f0-de1d720cf2ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:07.300111 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671dddd6-54f9-4edf-80f0-de1d720cf2ca None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:07.300424 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671dddd6-54f9-4edf-80f0-de1d720cf2ca None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:07.301148 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671dddd6-54f9-4edf-80f0-de1d720cf2ca None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:07.301911 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671dddd6-54f9-4edf-80f0-de1d720cf2ca None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:07.303649 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671dddd6-54f9-4edf-80f0-de1d720cf2ca None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:07.328093 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3958ac7-f5bf-4ec5-8a1e-8b2015720bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:07.328761 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3958ac7-f5bf-4ec5-8a1e-8b2015720bbe None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:07.329044 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3958ac7-f5bf-4ec5-8a1e-8b2015720bbe None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:07.329319 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3958ac7-f5bf-4ec5-8a1e-8b2015720bbe None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:07.330055 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3958ac7-f5bf-4ec5-8a1e-8b2015720bbe None admin] RBAC: Adding query filter params (name=compute) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:16:07.330362 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3958ac7-f5bf-4ec5-8a1e-8b2015720bbe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:07.332419 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3958ac7-f5bf-4ec5-8a1e-8b2015720bbe None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:07.337398 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 510/1017] 10.222.0.33 () {56 vars in 1112 bytes} [Fri Jul 15 19:16:07 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:16:07.342861 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3958ac7-f5bf-4ec5-8a1e-8b2015720bbe None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:07.419751 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:07.420620 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 508/1018] 10.222.0.33 () {60 vars in 1141 bytes} [Fri Jul 15 19:16:07 2022] POST /identity/v3/services => generated 242 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:16:07.446692 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3958ac7-f5bf-4ec5-8a1e-8b2015720bbe None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:07.494234 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f86fef5-0237-4eda-bfc2-6e2da7233a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:07.494920 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f86fef5-0237-4eda-bfc2-6e2da7233a81 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:07.495201 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f86fef5-0237-4eda-bfc2-6e2da7233a81 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:07.495480 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f86fef5-0237-4eda-bfc2-6e2da7233a81 None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:07.496191 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f86fef5-0237-4eda-bfc2-6e2da7233a81 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:16:07.496495 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f86fef5-0237-4eda-bfc2-6e2da7233a81 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:07.498114 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f86fef5-0237-4eda-bfc2-6e2da7233a81 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:07.505836 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 511/1019] 10.222.0.33 () {56 vars in 1112 bytes} [Fri Jul 15 19:16:07 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:16:07.510504 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671dddd6-54f9-4edf-80f0-de1d720cf2ca None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:07.544815 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:07.625438 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671dddd6-54f9-4edf-80f0-de1d720cf2ca None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:07.661465 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb43ffb0-34dd-4253-baf0-9f0ca63bf431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:07.662164 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb43ffb0-34dd-4253-baf0-9f0ca63bf431 None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:07.662448 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb43ffb0-34dd-4253-baf0-9f0ca63bf431 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:07.662723 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb43ffb0-34dd-4253-baf0-9f0ca63bf431 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:07.663475 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb43ffb0-34dd-4253-baf0-9f0ca63bf431 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:07.664899 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb43ffb0-34dd-4253-baf0-9f0ca63bf431 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:07.740285 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 509/1020] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 15 19:16:07 2022] POST /identity/v3/endpoints => generated 334 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:16:08.937846 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac87162d-7e23-44b8-a680-653531177f6f None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:08.938416 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac87162d-7e23-44b8-a680-653531177f6f None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:08.938416 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac87162d-7e23-44b8-a680-653531177f6f None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:08.939932 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 512/1021] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:16:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 19:16:08.945418 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467478f5-1526-4305-bbb0-ada585b90f33 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:08.945938 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467478f5-1526-4305-bbb0-ada585b90f33 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:08.946014 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467478f5-1526-4305-bbb0-ada585b90f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:08.962747 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-467478f5-1526-4305-bbb0-ada585b90f33 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:16:08.963935 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-467478f5-1526-4305-bbb0-ada585b90f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:16:09.039846 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 510/1022] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:16:08 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:16:09.064113 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac87162d-7e23-44b8-a680-653531177f6f None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:09.170067 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac87162d-7e23-44b8-a680-653531177f6f None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:09.204506 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a33080-04ca-4a06-89cb-0fde534fc7ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:09.205302 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a33080-04ca-4a06-89cb-0fde534fc7ba None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:09.205587 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a33080-04ca-4a06-89cb-0fde534fc7ba None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:09.206374 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a33080-04ca-4a06-89cb-0fde534fc7ba None admin] PATH_INFO: `/v3/domains/default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:09.207564 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a33080-04ca-4a06-89cb-0fde534fc7ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:09.208980 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a33080-04ca-4a06-89cb-0fde534fc7ba None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:09.210799 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 513/1023] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:16:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:16:09.216466 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467478f5-1526-4305-bbb0-ada585b90f33 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:09.231081 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:09.319321 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467478f5-1526-4305-bbb0-ada585b90f33 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:09.355903 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0ea863-6356-4102-84e9-863dc7f1ce8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:09.356909 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ea863-6356-4102-84e9-863dc7f1ce8a None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:09.357374 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ea863-6356-4102-84e9-863dc7f1ce8a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:09.357859 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ea863-6356-4102-84e9-863dc7f1ce8a None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:09.358768 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0ea863-6356-4102-84e9-863dc7f1ce8a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:09.360413 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0ea863-6356-4102-84e9-863dc7f1ce8a None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:09.493794 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 511/1024] 10.222.0.33 () {60 vars in 1132 bytes} [Fri Jul 15 19:16:09 2022] POST /identity/v3/users => generated 285 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:16:09.501158 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f57587-2e58-4081-86d8-27c3c6cf4b5e None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:09.501475 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f57587-2e58-4081-86d8-27c3c6cf4b5e None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:09.501728 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f57587-2e58-4081-86d8-27c3c6cf4b5e None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:09.504027 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 514/1025] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:16:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:16:09.509707 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e175f846-75ea-403b-a821-9b0c49da9e71 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:09.510101 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e175f846-75ea-403b-a821-9b0c49da9e71 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:09.510370 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e175f846-75ea-403b-a821-9b0c49da9e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:09.529413 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-e175f846-75ea-403b-a821-9b0c49da9e71 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:16:09.530679 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e175f846-75ea-403b-a821-9b0c49da9e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:16:09.603803 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:09.610677 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 512/1026] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:16:09 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:16:09.634212 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f57587-2e58-4081-86d8-27c3c6cf4b5e None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:09.639268 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:09.744445 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f57587-2e58-4081-86d8-27c3c6cf4b5e None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:09.781613 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68eebcc-6db0-462f-866d-38103d46ebc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:09.782603 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68eebcc-6db0-462f-866d-38103d46ebc2 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:09.782889 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68eebcc-6db0-462f-866d-38103d46ebc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:09.783159 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68eebcc-6db0-462f-866d-38103d46ebc2 None admin] PATH_INFO: `/v3/services/image` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:09.783812 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68eebcc-6db0-462f-866d-38103d46ebc2 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:09.785221 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68eebcc-6db0-462f-866d-38103d46ebc2 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:09.796894 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-d68eebcc-6db0-462f-866d-38103d46ebc2 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 15 19:16:09.796894 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:16:09.796894 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 19:16:09.796894 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 19:16:09.796894 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 19:16:09.796894 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 19:16:09.796894 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 19:16:09.796894 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:16:09.796894 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 15 19:16:09.796894 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:16:09.796894 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 19:16:09.796894 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:16:09.796894 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:16:09.796894 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:16:09.796894 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:16:09.796894 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:16:09.796894 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:16:09.797915 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:16:09.797915 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:16:09.797915 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:16:09.797915 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:16:09.797915 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:16:09.797915 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:16:09.797915 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 19:16:09.797915 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 19:16:09.797915 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 19:16:09.797915 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 19:16:09.797915 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:16:09.797915 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:16:09.797915 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:16:09.797915 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:16:09.797915 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:16:09.797915 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:16:09.798779 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:16:09.798779 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:16:09.798779 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:16:09.798779 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:16:09.798779 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:16:09.798779 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:16:09.798779 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:16:09.798779 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:16:09.798779 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:16:09.798779 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:16:09.798779 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 19:16:09.798779 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:16:09.798779 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 15 19:16:09.798779 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:16:09.798779 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 515/1027] 10.222.0.33 () {56 vars in 1105 bytes} [Fri Jul 15 19:16:09 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:16:09.804292 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e175f846-75ea-403b-a821-9b0c49da9e71 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:09.844054 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11e2120-6f97-42a3-8e98-55b4d3f00e64 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:09.844368 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11e2120-6f97-42a3-8e98-55b4d3f00e64 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:09.844624 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11e2120-6f97-42a3-8e98-55b4d3f00e64 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:09.846222 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 516/1028] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:16:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 19:16:09.851409 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb282af-aa51-436f-b58f-1b580dd0fa13 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:09.851700 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb282af-aa51-436f-b58f-1b580dd0fa13 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:09.851981 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb282af-aa51-436f-b58f-1b580dd0fa13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:09.869136 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-ceb282af-aa51-436f-b58f-1b580dd0fa13 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:16:09.870289 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-ceb282af-aa51-436f-b58f-1b580dd0fa13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:16:09.914340 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e175f846-75ea-403b-a821-9b0c49da9e71 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:09.942022 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:09.950970 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 517/1029] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:16:09 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 19:16:09.957055 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d25c609-d972-4e4a-93de-1ee98c90da02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:09.957959 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25c609-d972-4e4a-93de-1ee98c90da02 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:09.961010 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25c609-d972-4e4a-93de-1ee98c90da02 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:09.961010 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25c609-d972-4e4a-93de-1ee98c90da02 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:09.961010 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d25c609-d972-4e4a-93de-1ee98c90da02 None admin] RBAC: Adding query filter params (name=image) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:16:09.961010 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d25c609-d972-4e4a-93de-1ee98c90da02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:09.961010 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d25c609-d972-4e4a-93de-1ee98c90da02 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:09.972362 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 513/1030] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:16:09 2022] GET /identity/v3/services?name=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:16:09.973455 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb282af-aa51-436f-b58f-1b580dd0fa13 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:09.974886 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d25c609-d972-4e4a-93de-1ee98c90da02 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:10.047140 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:10.078781 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb282af-aa51-436f-b58f-1b580dd0fa13 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:10.090094 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d25c609-d972-4e4a-93de-1ee98c90da02 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:10.114823 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10dcdeb-f3ca-41a1-a866-64517f596502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:10.115607 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10dcdeb-f3ca-41a1-a866-64517f596502 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:10.115958 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10dcdeb-f3ca-41a1-a866-64517f596502 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:10.116233 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10dcdeb-f3ca-41a1-a866-64517f596502 None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:10.116955 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10dcdeb-f3ca-41a1-a866-64517f596502 None admin] RBAC: Adding query filter params (type=image) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:16:10.117262 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10dcdeb-f3ca-41a1-a866-64517f596502 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:10.118664 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10dcdeb-f3ca-41a1-a866-64517f596502 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:10.123704 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 518/1031] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:16:09 2022] GET /identity/v3/services?type=image => generated 350 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:16:10.127986 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467a1e52-05e8-4fc5-96b0-8981ccb9b6e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:10.128995 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467a1e52-05e8-4fc5-96b0-8981ccb9b6e3 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:10.129279 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467a1e52-05e8-4fc5-96b0-8981ccb9b6e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:10.129390 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10dcdeb-f3ca-41a1-a866-64517f596502 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:10.129548 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467a1e52-05e8-4fc5-96b0-8981ccb9b6e3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:10.145609 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467a1e52-05e8-4fc5-96b0-8981ccb9b6e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:16:10.153795 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-467a1e52-05e8-4fc5-96b0-8981ccb9b6e3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 15 19:16:10.153795 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:16:10.153795 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:16:10.153795 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:16:10.153795 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:16:10.153795 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:16:10.153795 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:16:10.153795 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:16:10.153795 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:16:10.153795 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:16:10.153795 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:16:10.153795 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:16:10.153795 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:16:10.153795 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:16:10.153795 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:16:10.153795 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:16:10.154769 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:16:10.154769 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:16:10.154769 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:16:10.154769 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:16:10.154769 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:16:10.154769 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:16:10.154769 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:16:10.154769 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:16:10.154769 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:16:10.154769 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:16:10.154769 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:16:10.154769 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:16:10.154769 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:16:10.154769 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:16:10.154769 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:16:10.154769 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:16:10.155668 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:16:10.155668 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:16:10.155668 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:16:10.155668 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:16:10.155668 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:16:10.155668 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:16:10.155668 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:16:10.155668 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:16:10.155668 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:16:10.155668 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:16:10.155668 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 15 19:16:10.155668 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:16:10.155668 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 514/1032] 10.222.0.33 () {56 vars in 1120 bytes} [Fri Jul 15 19:16:09 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:16:10.161804 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467a1e52-05e8-4fc5-96b0-8981ccb9b6e3 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:10.195329 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:10.236514 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10dcdeb-f3ca-41a1-a866-64517f596502 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:10.276625 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5457d7df-32db-42e2-9f8d-bb9f9de24705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:10.277636 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5457d7df-32db-42e2-9f8d-bb9f9de24705 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:10.278140 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5457d7df-32db-42e2-9f8d-bb9f9de24705 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:10.278595 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467a1e52-05e8-4fc5-96b0-8981ccb9b6e3 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:10.278801 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5457d7df-32db-42e2-9f8d-bb9f9de24705 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:10.279822 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5457d7df-32db-42e2-9f8d-bb9f9de24705 None admin] RBAC: Adding query filter params (interface=public, service_id=55992ba9600347d8a7304ea9509dbbf7, region_id=RegionOne) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:16:10.280436 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5457d7df-32db-42e2-9f8d-bb9f9de24705 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:10.282199 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5457d7df-32db-42e2-9f8d-bb9f9de24705 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:10.290839 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 519/1033] 10.222.0.33 () {56 vars in 1251 bytes} [Fri Jul 15 19:16:10 2022] GET /identity/v3/endpoints?service_id=55992ba9600347d8a7304ea9509dbbf7&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:16:10.296828 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5457d7df-32db-42e2-9f8d-bb9f9de24705 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:10.316589 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:10.331393 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78335e46-c439-4616-9901-cd4401bca813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:10.332073 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78335e46-c439-4616-9901-cd4401bca813 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:10.332416 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78335e46-c439-4616-9901-cd4401bca813 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:10.332756 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78335e46-c439-4616-9901-cd4401bca813 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:10.333615 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78335e46-c439-4616-9901-cd4401bca813 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:16:10.334259 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78335e46-c439-4616-9901-cd4401bca813 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:10.335879 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78335e46-c439-4616-9901-cd4401bca813 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:10.346073 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 515/1034] 10.222.0.33 () {56 vars in 1115 bytes} [Fri Jul 15 19:16:10 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:16:10.352140 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78335e46-c439-4616-9901-cd4401bca813 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:10.407406 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5457d7df-32db-42e2-9f8d-bb9f9de24705 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:10.434726 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:10.443728 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d4e164-563c-49c2-8f26-42707f32fbc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:10.444391 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d4e164-563c-49c2-8f26-42707f32fbc3 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:10.444674 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d4e164-563c-49c2-8f26-42707f32fbc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:10.444952 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d4e164-563c-49c2-8f26-42707f32fbc3 None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:10.445739 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d4e164-563c-49c2-8f26-42707f32fbc3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:10.447121 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d4e164-563c-49c2-8f26-42707f32fbc3 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:10.454295 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 520/1035] 10.222.0.33 () {56 vars in 1087 bytes} [Fri Jul 15 19:16:10 2022] GET /identity/v3/services => generated 1957 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 19:16:10.466942 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78335e46-c439-4616-9901-cd4401bca813 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:10.507000 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801598c8-3089-4c49-8efe-63ec8a408c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:10.507835 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801598c8-3089-4c49-8efe-63ec8a408c59 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:10.508127 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801598c8-3089-4c49-8efe-63ec8a408c59 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:10.508392 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801598c8-3089-4c49-8efe-63ec8a408c59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:10.520373 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801598c8-3089-4c49-8efe-63ec8a408c59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:10.521917 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801598c8-3089-4c49-8efe-63ec8a408c59 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:10.541392 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-801598c8-3089-4c49-8efe-63ec8a408c59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:16:10.541392 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:16:10.541392 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:16:10.541392 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:16:10.541392 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:16:10.541392 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:16:10.541392 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:16:10.541392 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:16:10.541392 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:16:10.541392 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:16:10.541392 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:16:10.541392 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:16:10.541392 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:16:10.541392 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:16:10.541392 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:16:10.542449 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:16:10.542449 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:16:10.542449 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:16:10.542449 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:16:10.542449 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:16:10.542449 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:16:10.542449 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:16:10.542449 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:16:10.542449 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:16:10.542449 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:16:10.542449 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:16:10.542449 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:16:10.542449 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:16:10.542449 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:16:10.542449 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:16:10.542449 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:16:10.542449 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:16:10.543381 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:16:10.543381 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:16:10.543381 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:16:10.543381 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:16:10.546123 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 516/1036] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:16:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:16:10.548690 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d4e164-563c-49c2-8f26-42707f32fbc3 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:10.666410 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d4e164-563c-49c2-8f26-42707f32fbc3 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:10.699548 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2cb2ea-c04f-4846-9cd5-e942915f4cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:10.700213 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2cb2ea-c04f-4846-9cd5-e942915f4cd7 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:10.700494 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2cb2ea-c04f-4846-9cd5-e942915f4cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:10.700770 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2cb2ea-c04f-4846-9cd5-e942915f4cd7 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:10.701490 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2cb2ea-c04f-4846-9cd5-e942915f4cd7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:16:10.701824 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2cb2ea-c04f-4846-9cd5-e942915f4cd7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:10.703194 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2cb2ea-c04f-4846-9cd5-e942915f4cd7 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:10.720773 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 521/1037] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:16:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:16:10.723281 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801598c8-3089-4c49-8efe-63ec8a408c59 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:10.760028 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:10.836416 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801598c8-3089-4c49-8efe-63ec8a408c59 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:10.870995 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84475ad7-af93-4ca1-ac89-cb5f8b489db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:10.871778 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84475ad7-af93-4ca1-ac89-cb5f8b489db4 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:10.872058 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84475ad7-af93-4ca1-ac89-cb5f8b489db4 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:10.872332 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84475ad7-af93-4ca1-ac89-cb5f8b489db4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:10.879542 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84475ad7-af93-4ca1-ac89-cb5f8b489db4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:10.880983 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84475ad7-af93-4ca1-ac89-cb5f8b489db4 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:10.895888 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-84475ad7-af93-4ca1-ac89-cb5f8b489db4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 15 19:16:10.895888 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:16:10.895888 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:16:10.895888 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:16:10.895888 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:16:10.895888 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:16:10.895888 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:16:10.895888 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:16:10.895888 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:16:10.895888 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:16:10.895888 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:16:10.895888 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:16:10.895888 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 19:16:10.895888 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 19:16:10.895888 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 19:16:10.895888 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 19:16:10.896911 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:16:10.896911 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:16:10.896911 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:16:10.896911 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:16:10.896911 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:16:10.896911 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:16:10.896911 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:16:10.896911 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:16:10.896911 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:16:10.896911 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:16:10.896911 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:16:10.896911 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:16:10.896911 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:16:10.896911 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:16:10.896911 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:16:10.896911 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:16:10.896911 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:16:10.896911 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:16:10.897973 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:16:10.897973 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:16:10.897973 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 19:16:10.897973 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 19:16:10.897973 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 19:16:10.897973 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 19:16:10.897973 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 19:16:10.897973 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 19:16:10.897973 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 15 19:16:10.897973 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:16:10.897973 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 517/1038] 10.222.0.33 () {56 vars in 1093 bytes} [Fri Jul 15 19:16:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:16:10.903013 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2cb2ea-c04f-4846-9cd5-e942915f4cd7 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:10.956147 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:11.007396 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2cb2ea-c04f-4846-9cd5-e942915f4cd7 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:11.041912 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47f8539-fff5-41e1-ae25-6b06bca5f4f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:11.042585 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47f8539-fff5-41e1-ae25-6b06bca5f4f1 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:11.042869 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47f8539-fff5-41e1-ae25-6b06bca5f4f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:11.043149 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47f8539-fff5-41e1-ae25-6b06bca5f4f1 None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:11.043986 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47f8539-fff5-41e1-ae25-6b06bca5f4f1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:16:11.044295 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47f8539-fff5-41e1-ae25-6b06bca5f4f1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:11.045722 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47f8539-fff5-41e1-ae25-6b06bca5f4f1 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:11.063650 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 522/1039] 10.222.0.33 () {56 vars in 1133 bytes} [Fri Jul 15 19:16:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:16:11.069341 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84475ad7-af93-4ca1-ac89-cb5f8b489db4 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:11.165787 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:11.171977 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84475ad7-af93-4ca1-ac89-cb5f8b489db4 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:11.206362 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d628c4-28f9-41ef-973b-f03293cec7b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:11.206935 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d628c4-28f9-41ef-973b-f03293cec7b1 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:11.207218 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d628c4-28f9-41ef-973b-f03293cec7b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:11.207494 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d628c4-28f9-41ef-973b-f03293cec7b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:11.275404 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d628c4-28f9-41ef-973b-f03293cec7b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:11.276857 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d628c4-28f9-41ef-973b-f03293cec7b1 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:11.369832 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:11.376950 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 518/1040] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 15 19:16:11 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 19:16:11.382650 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47f8539-fff5-41e1-ae25-6b06bca5f4f1 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:11.487342 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47f8539-fff5-41e1-ae25-6b06bca5f4f1 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:11.496635 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:11.522172 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d046e9-fe1e-4174-9b5b-20310b53fbdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:11.522965 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d046e9-fe1e-4174-9b5b-20310b53fbdb None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:11.523248 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d046e9-fe1e-4174-9b5b-20310b53fbdb None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:11.523525 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d046e9-fe1e-4174-9b5b-20310b53fbdb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:11.535095 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d046e9-fe1e-4174-9b5b-20310b53fbdb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:11.536549 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d046e9-fe1e-4174-9b5b-20310b53fbdb None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:11.554265 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-d0d046e9-fe1e-4174-9b5b-20310b53fbdb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:16:11.554265 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:16:11.554265 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:16:11.554265 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:16:11.554265 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:16:11.554265 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:16:11.554265 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:16:11.554265 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:16:11.554265 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:16:11.554265 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:16:11.554265 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:16:11.554265 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:16:11.554265 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:16:11.554265 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:16:11.554265 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:16:11.555200 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:16:11.555200 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:16:11.555200 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:16:11.555200 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:16:11.555200 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:16:11.555200 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:16:11.555200 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:16:11.555200 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:16:11.555200 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:16:11.555200 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:16:11.555200 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:16:11.555200 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:16:11.555200 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:16:11.555200 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:16:11.555200 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:16:11.555200 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:16:11.555200 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:16:11.560828 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:16:11.560828 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:16:11.560828 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:16:11.560828 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:16:11.560828 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 523/1041] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:16:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:16:11.562684 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d628c4-28f9-41ef-973b-f03293cec7b1 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:11.616892 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0894f4d-b043-4d56-a0ba-d30653f42efc None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:11.617207 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0894f4d-b043-4d56-a0ba-d30653f42efc None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:11.617455 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0894f4d-b043-4d56-a0ba-d30653f42efc None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:11.619016 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 524/1042] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:16:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 19:16:11.623785 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc64c33-0221-47b3-8aa0-87bdbb6c1904 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:11.624074 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc64c33-0221-47b3-8aa0-87bdbb6c1904 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:11.624357 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc64c33-0221-47b3-8aa0-87bdbb6c1904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:11.641103 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-6cc64c33-0221-47b3-8aa0-87bdbb6c1904 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:16:11.642322 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-6cc64c33-0221-47b3-8aa0-87bdbb6c1904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:16:11.676528 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d628c4-28f9-41ef-973b-f03293cec7b1 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:11.713479 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 525/1043] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:16:11 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 19:16:11.725009 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da71f42b-cd0a-4fca-ba4e-f0cee2e08e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:11.725709 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da71f42b-cd0a-4fca-ba4e-f0cee2e08e35 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:11.726036 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da71f42b-cd0a-4fca-ba4e-f0cee2e08e35 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:11.726314 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da71f42b-cd0a-4fca-ba4e-f0cee2e08e35 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:11.727052 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da71f42b-cd0a-4fca-ba4e-f0cee2e08e35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:16:11.727363 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da71f42b-cd0a-4fca-ba4e-f0cee2e08e35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:11.728731 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da71f42b-cd0a-4fca-ba4e-f0cee2e08e35 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:11.737890 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc64c33-0221-47b3-8aa0-87bdbb6c1904 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:11.740807 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 519/1044] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:16:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:16:11.746709 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da71f42b-cd0a-4fca-ba4e-f0cee2e08e35 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:11.812096 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:11.849094 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:11.858658 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da71f42b-cd0a-4fca-ba4e-f0cee2e08e35 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:11.859475 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc64c33-0221-47b3-8aa0-87bdbb6c1904 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:11.895568 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6628752e-b153-4848-a425-59082910a950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:11.896392 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6628752e-b153-4848-a425-59082910a950 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:11.896670 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6628752e-b153-4848-a425-59082910a950 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:11.896943 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6628752e-b153-4848-a425-59082910a950 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:11.900390 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c03b50a-13f6-4ac7-a13c-9804d4765b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:11.902272 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c03b50a-13f6-4ac7-a13c-9804d4765b80 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:11.902560 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c03b50a-13f6-4ac7-a13c-9804d4765b80 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:11.902836 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c03b50a-13f6-4ac7-a13c-9804d4765b80 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:11.904144 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6628752e-b153-4848-a425-59082910a950 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:11.905966 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6628752e-b153-4848-a425-59082910a950 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:11.917802 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c03b50a-13f6-4ac7-a13c-9804d4765b80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:16:11.922519 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-6628752e-b153-4848-a425-59082910a950 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:16:11.922519 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:16:11.922519 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:16:11.922519 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:16:11.922519 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:16:11.922519 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:16:11.922519 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:16:11.922519 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:16:11.922519 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:16:11.922519 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:16:11.922519 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:16:11.922519 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:16:11.922519 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 19:16:11.922519 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 19:16:11.922519 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 19:16:11.923426 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 19:16:11.923426 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:16:11.923426 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:16:11.923426 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:16:11.923426 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:16:11.923426 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:16:11.923426 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:16:11.923426 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:16:11.923426 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:16:11.923426 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:16:11.923426 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:16:11.923426 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:16:11.923426 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:16:11.923426 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:16:11.923426 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:16:11.923426 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:16:11.923426 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:16:11.924349 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 19:16:11.924349 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 19:16:11.924349 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:16:11.924349 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:16:11.924349 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:16:11.924349 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:16:11.924349 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:16:11.924349 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:16:11.926148 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 520/1045] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 15 19:16:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:16:11.926336 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-7c03b50a-13f6-4ac7-a13c-9804d4765b80 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:16:11.926336 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:16:11.926336 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:16:11.926336 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:16:11.926336 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:16:11.926336 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:16:11.926336 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:16:11.926336 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:16:11.926336 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:16:11.926336 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:16:11.926336 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:16:11.926336 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:16:11.926336 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:16:11.926336 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:16:11.926336 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:16:11.926336 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:16:11.927447 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:16:11.927447 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:16:11.927447 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:16:11.927447 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:16:11.927447 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:16:11.927447 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:16:11.927447 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:16:11.927447 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:16:11.927447 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:16:11.927447 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:16:11.927447 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:16:11.927447 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:16:11.927447 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:16:11.927447 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:16:11.927447 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:16:11.927447 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:16:11.928310 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:16:11.928310 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:16:11.928310 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:16:11.928310 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:16:11.928310 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:16:11.928310 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:16:11.928310 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:16:11.928310 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:16:11.928310 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:16:11.928310 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:16:11.928310 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:16:11.928310 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:16:11.928310 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 526/1046] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jul 15 19:16:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:16:11.930535 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c03b50a-13f6-4ac7-a13c-9804d4765b80 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:11.934609 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6628752e-b153-4848-a425-59082910a950 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:12.068560 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6628752e-b153-4848-a425-59082910a950 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:12.072782 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c03b50a-13f6-4ac7-a13c-9804d4765b80 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:12.094617 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:12.110824 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f4a88c-669f-426a-ad32-554353d5eadb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:12.111490 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f4a88c-669f-426a-ad32-554353d5eadb None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:12.111854 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f4a88c-669f-426a-ad32-554353d5eadb None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:12.112853 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f4a88c-669f-426a-ad32-554353d5eadb None admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:12.113738 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f4a88c-669f-426a-ad32-554353d5eadb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:16:12.114315 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f4a88c-669f-426a-ad32-554353d5eadb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:12.116530 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f4a88c-669f-426a-ad32-554353d5eadb None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:12.117518 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1071122b-f652-48b5-a47c-5a31c448a49d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:12.118689 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1071122b-f652-48b5-a47c-5a31c448a49d None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:12.119177 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1071122b-f652-48b5-a47c-5a31c448a49d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:12.120029 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1071122b-f652-48b5-a47c-5a31c448a49d None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:12.120948 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1071122b-f652-48b5-a47c-5a31c448a49d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:16:12.121402 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1071122b-f652-48b5-a47c-5a31c448a49d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:12.122954 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1071122b-f652-48b5-a47c-5a31c448a49d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:12.134995 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 527/1047] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jul 15 19:16:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:16:12.137134 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 521/1048] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 15 19:16:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:16:12.137253 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f4a88c-669f-426a-ad32-554353d5eadb None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:12.142235 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1071122b-f652-48b5-a47c-5a31c448a49d None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:12.151664 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:12.251796 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f4a88c-669f-426a-ad32-554353d5eadb None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:12.252006 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1071122b-f652-48b5-a47c-5a31c448a49d None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:12.288218 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c959f3-fadb-43df-8d21-c85794a9a3fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:12.288445 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153cad1c-8ac9-4412-aa64-1b203fa405c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:12.289066 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c959f3-fadb-43df-8d21-c85794a9a3fb None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:12.289378 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153cad1c-8ac9-4412-aa64-1b203fa405c9 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:12.289502 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c959f3-fadb-43df-8d21-c85794a9a3fb None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:12.289806 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153cad1c-8ac9-4412-aa64-1b203fa405c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:12.289904 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c959f3-fadb-43df-8d21-c85794a9a3fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:12.290207 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153cad1c-8ac9-4412-aa64-1b203fa405c9 None admin] PATH_INFO: `/v3/users/5f76859b74f14fc499f2c658439e8129` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:12.290829 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c959f3-fadb-43df-8d21-c85794a9a3fb None admin] RBAC: Adding query filter params (role.id=c1dc7198be2d43d3aa44d16107d9a959, scope.project.id=55a756ccaef14cb49b26b0d68dd0b466, user.id=7673a3b5c0324e07a9535ac70ed7a491) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:16:12.291144 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c959f3-fadb-43df-8d21-c85794a9a3fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:12.292650 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c959f3-fadb-43df-8d21-c85794a9a3fb None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:12.298562 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 528/1049] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 15 19:16:12 2022] GET /identity/v3/role_assignments?user.id=7673a3b5c0324e07a9535ac70ed7a491&scope.project.id=55a756ccaef14cb49b26b0d68dd0b466&role.id=c1dc7198be2d43d3aa44d16107d9a959 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:16:12.308207 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153cad1c-8ac9-4412-aa64-1b203fa405c9 None admin] RBAC: Authorizing `identity:get_user(user_id=5f76859b74f14fc499f2c658439e8129)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:12.309797 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153cad1c-8ac9-4412-aa64-1b203fa405c9 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:12.311900 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 522/1050] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 15 19:16:12 2022] GET /identity/v3/users/5f76859b74f14fc499f2c658439e8129 => generated 285 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:16:12.318829 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c959f3-fadb-43df-8d21-c85794a9a3fb None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:12.418865 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:12.451168 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c959f3-fadb-43df-8d21-c85794a9a3fb None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:12.489864 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade73f45-575b-4afc-a1b2-38b036e417da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:12.490522 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade73f45-575b-4afc-a1b2-38b036e417da None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:12.490805 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade73f45-575b-4afc-a1b2-38b036e417da None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:12.491078 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade73f45-575b-4afc-a1b2-38b036e417da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:12.525604 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f295f12-8a05-4a4e-803f-9373081fa261 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:12.525933 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f295f12-8a05-4a4e-803f-9373081fa261 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:12.526216 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f295f12-8a05-4a4e-803f-9373081fa261 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:12.527614 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 523/1051] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:16:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:16:12.533400 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca3aa7-a8a6-4b81-b1c4-757ce3530882 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:12.533690 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca3aa7-a8a6-4b81-b1c4-757ce3530882 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:12.534585 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca3aa7-a8a6-4b81-b1c4-757ce3530882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:12.552667 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-2eca3aa7-a8a6-4b81-b1c4-757ce3530882 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:16:12.553916 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-2eca3aa7-a8a6-4b81-b1c4-757ce3530882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:16:12.567858 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade73f45-575b-4afc-a1b2-38b036e417da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:12.570253 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade73f45-575b-4afc-a1b2-38b036e417da None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:12.586323 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:12.632323 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 524/1052] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:16:12 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 19:16:12.652836 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:12.654860 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eca3aa7-a8a6-4b81-b1c4-757ce3530882 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:12.682532 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 529/1053] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 15 19:16:12 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 19:16:12.688144 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade73f45-575b-4afc-a1b2-38b036e417da None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:12.779625 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eca3aa7-a8a6-4b81-b1c4-757ce3530882 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:12.804587 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade73f45-575b-4afc-a1b2-38b036e417da None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:12.819294 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d506b51d-b359-40fb-ab3c-8815826dc0fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:12.820766 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d506b51d-b359-40fb-ab3c-8815826dc0fe None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:12.821204 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d506b51d-b359-40fb-ab3c-8815826dc0fe None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:12.821622 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d506b51d-b359-40fb-ab3c-8815826dc0fe None admin] PATH_INFO: `/v3/services/image` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:12.822459 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d506b51d-b359-40fb-ab3c-8815826dc0fe None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:12.823968 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d506b51d-b359-40fb-ab3c-8815826dc0fe None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:12.836253 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-d506b51d-b359-40fb-ab3c-8815826dc0fe None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 15 19:16:12.836253 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:16:12.836253 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 19:16:12.836253 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 19:16:12.836253 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 19:16:12.836253 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 19:16:12.836253 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 19:16:12.836253 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:16:12.836253 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 15 19:16:12.836253 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:16:12.836253 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 19:16:12.836253 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:16:12.836253 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:16:12.836253 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:16:12.836253 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:16:12.836253 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:16:12.836253 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:16:12.837422 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:16:12.837422 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:16:12.837422 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:16:12.837422 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:16:12.837422 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:16:12.837422 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:16:12.837422 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 19:16:12.837422 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 19:16:12.837422 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 19:16:12.837422 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 19:16:12.837422 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:16:12.837422 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:16:12.837422 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:16:12.837422 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:16:12.837422 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:16:12.837422 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:16:12.838463 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:16:12.838463 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:16:12.838463 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:16:12.838463 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:16:12.838463 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:16:12.838463 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:16:12.838463 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:16:12.838463 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:16:12.838463 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:16:12.838463 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:16:12.838463 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 19:16:12.838463 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:16:12.838463 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 15 19:16:12.838463 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:16:12.840306 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95843296-21aa-4935-847f-2a9b3bd6f3ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:12.841115 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 525/1054] 10.222.0.33 () {56 vars in 1105 bytes} [Fri Jul 15 19:16:12 2022] GET /identity/v3/services/image => generated 86 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:16:12.841221 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95843296-21aa-4935-847f-2a9b3bd6f3ba None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:12.841443 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95843296-21aa-4935-847f-2a9b3bd6f3ba None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:12.841717 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95843296-21aa-4935-847f-2a9b3bd6f3ba None admin] PATH_INFO: `/v3/projects/ab5433faa0b14ec29f68a8b8219b16ed` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:12.842935 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95843296-21aa-4935-847f-2a9b3bd6f3ba None admin] RBAC: Authorizing `identity:get_project(project_id=ab5433faa0b14ec29f68a8b8219b16ed)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:12.844352 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95843296-21aa-4935-847f-2a9b3bd6f3ba None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:12.849798 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 530/1055] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 15 19:16:12 2022] GET /identity/v3/projects/ab5433faa0b14ec29f68a8b8219b16ed => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:16:12.850696 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d506b51d-b359-40fb-ab3c-8815826dc0fe None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:12.852602 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95843296-21aa-4935-847f-2a9b3bd6f3ba None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:12.883213 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:12.898910 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:12.959360 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d506b51d-b359-40fb-ab3c-8815826dc0fe None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:12.960205 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95843296-21aa-4935-847f-2a9b3bd6f3ba None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:12.995611 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc9d7b0-f9e4-48b7-8001-0d5e7a3bddb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:12.996370 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc9d7b0-f9e4-48b7-8001-0d5e7a3bddb1 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:12.996649 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc9d7b0-f9e4-48b7-8001-0d5e7a3bddb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:12.996923 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc9d7b0-f9e4-48b7-8001-0d5e7a3bddb1 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:12.997844 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc9d7b0-f9e4-48b7-8001-0d5e7a3bddb1 None admin] RBAC: Adding query filter params (name=image) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:16:12.998160 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc9d7b0-f9e4-48b7-8001-0d5e7a3bddb1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:12.998966 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0d5f03-00d4-4461-a623-4050e108b169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:12.999525 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc9d7b0-f9e4-48b7-8001-0d5e7a3bddb1 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:12.999625 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0d5f03-00d4-4461-a623-4050e108b169 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:12.999878 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0d5f03-00d4-4461-a623-4050e108b169 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:13.000147 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0d5f03-00d4-4461-a623-4050e108b169 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:13.000971 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0d5f03-00d4-4461-a623-4050e108b169 None admin] RBAC: Adding query filter params (role.id=4318f86ba9f74cd4a0bef1c311276657, scope.project.id=ab5433faa0b14ec29f68a8b8219b16ed, user.id=5f76859b74f14fc499f2c658439e8129) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:16:13.001280 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0d5f03-00d4-4461-a623-4050e108b169 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:13.002844 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0d5f03-00d4-4461-a623-4050e108b169 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:13.009737 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 531/1056] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 15 19:16:12 2022] GET /identity/v3/role_assignments?user.id=5f76859b74f14fc499f2c658439e8129&scope.project.id=ab5433faa0b14ec29f68a8b8219b16ed&role.id=4318f86ba9f74cd4a0bef1c311276657 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:16:13.010076 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 526/1057] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:16:12 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:16:13.011203 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc9d7b0-f9e4-48b7-8001-0d5e7a3bddb1 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:13.117261 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:13.125086 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc9d7b0-f9e4-48b7-8001-0d5e7a3bddb1 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:13.161672 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51c7cd9-a1f6-4098-8f6d-edba8f281d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:13.162708 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51c7cd9-a1f6-4098-8f6d-edba8f281d58 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:13.163008 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51c7cd9-a1f6-4098-8f6d-edba8f281d58 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:13.163294 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51c7cd9-a1f6-4098-8f6d-edba8f281d58 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:13.164014 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51c7cd9-a1f6-4098-8f6d-edba8f281d58 None admin] RBAC: Adding query filter params (type=image) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:16:13.164963 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51c7cd9-a1f6-4098-8f6d-edba8f281d58 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:13.165701 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51c7cd9-a1f6-4098-8f6d-edba8f281d58 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:13.171647 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 527/1058] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:16:13 2022] GET /identity/v3/services?type=image => generated 350 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:16:13.178485 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0d5f03-00d4-4461-a623-4050e108b169 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:13.286479 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0d5f03-00d4-4461-a623-4050e108b169 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:13.293418 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:13.324058 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74137a4-aa60-4ee6-9750-20c7ca84942f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:13.324759 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74137a4-aa60-4ee6-9750-20c7ca84942f None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:13.325041 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74137a4-aa60-4ee6-9750-20c7ca84942f None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:13.325312 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74137a4-aa60-4ee6-9750-20c7ca84942f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:13.326083 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74137a4-aa60-4ee6-9750-20c7ca84942f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:13.327450 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74137a4-aa60-4ee6-9750-20c7ca84942f None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:13.434029 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 532/1059] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 15 19:16:13 2022] POST /identity/v3/endpoints => generated 327 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:16:14.332632 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1a0950-fb93-4be4-82fa-969b1ace0620 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:14.336530 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1a0950-fb93-4be4-82fa-969b1ace0620 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:14.336530 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1a0950-fb93-4be4-82fa-969b1ace0620 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:14.336530 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 528/1060] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:16:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:16:14.340112 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b954527-a04a-451e-90fa-85f34a8f86d9 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:14.340543 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b954527-a04a-451e-90fa-85f34a8f86d9 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:14.340720 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b954527-a04a-451e-90fa-85f34a8f86d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:14.358711 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-9b954527-a04a-451e-90fa-85f34a8f86d9 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:16:14.359906 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-9b954527-a04a-451e-90fa-85f34a8f86d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:16:14.435041 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 533/1061] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:16:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:16:14.457485 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1a0950-fb93-4be4-82fa-969b1ace0620 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:14.571352 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1a0950-fb93-4be4-82fa-969b1ace0620 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:14.605914 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1f39da-c251-4d8e-b20d-c2495eb5efc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:14.606656 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1f39da-c251-4d8e-b20d-c2495eb5efc1 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:14.606939 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1f39da-c251-4d8e-b20d-c2495eb5efc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:14.607217 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1f39da-c251-4d8e-b20d-c2495eb5efc1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:14.619321 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1f39da-c251-4d8e-b20d-c2495eb5efc1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:16:14.627520 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-8a1f39da-c251-4d8e-b20d-c2495eb5efc1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 15 19:16:14.627520 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:16:14.627520 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:16:14.627520 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:16:14.627520 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:16:14.627520 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:16:14.627520 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:16:14.627520 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:16:14.627520 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:16:14.627520 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:16:14.627520 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:16:14.627520 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:16:14.627520 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:16:14.627520 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:16:14.627520 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:16:14.627520 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:16:14.628988 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:16:14.628988 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:16:14.628988 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:16:14.628988 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:16:14.628988 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:16:14.628988 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:16:14.628988 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:16:14.628988 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:16:14.628988 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:16:14.628988 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:16:14.628988 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:16:14.628988 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:16:14.628988 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:16:14.628988 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:16:14.628988 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:16:14.628988 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:16:14.630248 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:16:14.630248 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:16:14.630248 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:16:14.630248 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:16:14.630248 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:16:14.630248 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:16:14.630248 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:16:14.630248 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:16:14.630248 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:16:14.630248 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:16:14.630248 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 15 19:16:14.630248 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:16:14.630248 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 529/1062] 10.222.0.33 () {56 vars in 1120 bytes} [Fri Jul 15 19:16:14 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:16:14.635863 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b954527-a04a-451e-90fa-85f34a8f86d9 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:14.726142 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:14.738152 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b954527-a04a-451e-90fa-85f34a8f86d9 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:14.775670 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54714e7b-4348-40b2-b6b5-ae68b602ec1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:14.776746 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54714e7b-4348-40b2-b6b5-ae68b602ec1a None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:14.777244 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54714e7b-4348-40b2-b6b5-ae68b602ec1a None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:14.777735 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54714e7b-4348-40b2-b6b5-ae68b602ec1a None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:14.778521 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54714e7b-4348-40b2-b6b5-ae68b602ec1a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:16:14.778849 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54714e7b-4348-40b2-b6b5-ae68b602ec1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:14.780305 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54714e7b-4348-40b2-b6b5-ae68b602ec1a None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:14.793833 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 534/1063] 10.222.0.33 () {56 vars in 1115 bytes} [Fri Jul 15 19:16:14 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:16:14.796527 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1f39da-c251-4d8e-b20d-c2495eb5efc1 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:14.819164 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:14.909293 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1f39da-c251-4d8e-b20d-c2495eb5efc1 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:14.946882 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129a6d96-afc4-4e85-b7ac-48ccd309cfe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:14.948055 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129a6d96-afc4-4e85-b7ac-48ccd309cfe7 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:14.948547 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129a6d96-afc4-4e85-b7ac-48ccd309cfe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:14.949047 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129a6d96-afc4-4e85-b7ac-48ccd309cfe7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:14.960078 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129a6d96-afc4-4e85-b7ac-48ccd309cfe7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:14.961525 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129a6d96-afc4-4e85-b7ac-48ccd309cfe7 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:14.979444 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-129a6d96-afc4-4e85-b7ac-48ccd309cfe7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:16:14.979444 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:16:14.979444 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:16:14.979444 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:16:14.979444 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:16:14.979444 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:16:14.979444 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:16:14.979444 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:16:14.979444 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:16:14.979444 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:16:14.979444 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:16:14.979444 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:16:14.979444 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:16:14.979444 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:16:14.979444 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:16:14.980391 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:16:14.980391 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:16:14.980391 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:16:14.980391 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:16:14.980391 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:16:14.980391 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:16:14.980391 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:16:14.980391 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:16:14.980391 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:16:14.980391 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:16:14.980391 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:16:14.980391 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:16:14.980391 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:16:14.980391 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:16:14.980391 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:16:14.980391 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:16:14.980391 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:16:14.981321 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:16:14.981321 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:16:14.981321 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:16:14.981321 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:16:14.981321 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 530/1064] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:16:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:16:14.988448 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54714e7b-4348-40b2-b6b5-ae68b602ec1a None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:15.044291 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1c1880-a19a-4403-94a1-9c43e10666ae None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:15.044618 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1c1880-a19a-4403-94a1-9c43e10666ae None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:15.044854 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1c1880-a19a-4403-94a1-9c43e10666ae None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:15.046428 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 531/1065] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:16:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:16:15.051790 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45293a15-f425-43c1-ac7e-e4d6fd335224 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:15.052076 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45293a15-f425-43c1-ac7e-e4d6fd335224 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:15.052354 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45293a15-f425-43c1-ac7e-e4d6fd335224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:15.069174 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-45293a15-f425-43c1-ac7e-e4d6fd335224 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:16:15.070421 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-45293a15-f425-43c1-ac7e-e4d6fd335224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:16:15.096119 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54714e7b-4348-40b2-b6b5-ae68b602ec1a None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:15.131248 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c2dc57-1263-4948-a099-0a083d0dddcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:15.131907 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c2dc57-1263-4948-a099-0a083d0dddcc None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:15.132190 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c2dc57-1263-4948-a099-0a083d0dddcc None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:15.132460 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c2dc57-1263-4948-a099-0a083d0dddcc None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:15.133175 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c2dc57-1263-4948-a099-0a083d0dddcc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:16:15.133470 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c2dc57-1263-4948-a099-0a083d0dddcc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:15.134881 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c2dc57-1263-4948-a099-0a083d0dddcc None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:15.145502 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:15.152051 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 535/1066] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:16:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:16:15.152196 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 532/1067] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:16:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 19:16:15.153217 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c2dc57-1263-4948-a099-0a083d0dddcc None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:15.170327 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:15.175717 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45293a15-f425-43c1-ac7e-e4d6fd335224 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:15.262967 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c2dc57-1263-4948-a099-0a083d0dddcc None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:15.284866 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45293a15-f425-43c1-ac7e-e4d6fd335224 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:15.298508 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76e6a0e-72e4-4e62-98fd-567800630a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:15.299822 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e6a0e-72e4-4e62-98fd-567800630a8d None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:15.300115 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e6a0e-72e4-4e62-98fd-567800630a8d None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:15.300393 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e6a0e-72e4-4e62-98fd-567800630a8d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:15.307730 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76e6a0e-72e4-4e62-98fd-567800630a8d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:15.309168 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76e6a0e-72e4-4e62-98fd-567800630a8d None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:15.321225 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54d872f-1254-4c0e-8554-bc8761b529ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:15.322047 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d872f-1254-4c0e-8554-bc8761b529ba None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:15.322347 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d872f-1254-4c0e-8554-bc8761b529ba None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:15.322618 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d872f-1254-4c0e-8554-bc8761b529ba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:15.324226 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-a76e6a0e-72e4-4e62-98fd-567800630a8d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 15 19:16:15.324226 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:16:15.324226 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:16:15.324226 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:16:15.324226 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:16:15.324226 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:16:15.324226 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:16:15.324226 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:16:15.324226 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:16:15.324226 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:16:15.324226 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:16:15.324226 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:16:15.324226 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 19:16:15.324226 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 19:16:15.324226 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 19:16:15.324226 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 19:16:15.325334 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:16:15.325334 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:16:15.325334 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:16:15.325334 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:16:15.325334 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:16:15.325334 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:16:15.325334 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:16:15.325334 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:16:15.325334 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:16:15.325334 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:16:15.325334 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:16:15.325334 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:16:15.325334 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:16:15.325334 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:16:15.325334 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:16:15.325334 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:16:15.325334 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:16:15.325334 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:16:15.326346 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:16:15.326346 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:16:15.326346 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 19:16:15.326346 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 19:16:15.326346 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 19:16:15.326346 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 19:16:15.326346 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 19:16:15.326346 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 19:16:15.326346 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 15 19:16:15.326346 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:16:15.326346 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 536/1068] 10.222.0.33 () {56 vars in 1093 bytes} [Fri Jul 15 19:16:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:16:15.331672 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76e6a0e-72e4-4e62-98fd-567800630a8d None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:15.342347 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54d872f-1254-4c0e-8554-bc8761b529ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:16:15.350487 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-a54d872f-1254-4c0e-8554-bc8761b529ba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:16:15.350487 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:16:15.350487 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:16:15.350487 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:16:15.350487 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:16:15.350487 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:16:15.350487 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:16:15.350487 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:16:15.350487 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:16:15.350487 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:16:15.350487 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:16:15.350487 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:16:15.350487 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:16:15.350487 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:16:15.350487 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:16:15.350487 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:16:15.351528 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:16:15.351528 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:16:15.351528 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:16:15.351528 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:16:15.351528 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:16:15.351528 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:16:15.351528 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:16:15.351528 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:16:15.351528 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:16:15.351528 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:16:15.351528 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:16:15.351528 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:16:15.351528 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:16:15.351528 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:16:15.351528 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:16:15.351528 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:16:15.352398 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:16:15.352398 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:16:15.352398 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:16:15.352398 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:16:15.352398 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:16:15.352398 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:16:15.352398 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:16:15.352398 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:16:15.352398 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:16:15.352398 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:16:15.352398 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:16:15.352398 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:16:15.358104 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 533/1069] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jul 15 19:16:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:16:15.359383 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54d872f-1254-4c0e-8554-bc8761b529ba None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:15.432135 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:15.455254 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:15.464212 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76e6a0e-72e4-4e62-98fd-567800630a8d None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:15.500000 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4ba514-1175-40ce-a760-3fe5209bd85b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:15.500672 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4ba514-1175-40ce-a760-3fe5209bd85b None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:15.500959 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4ba514-1175-40ce-a760-3fe5209bd85b None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:15.501235 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4ba514-1175-40ce-a760-3fe5209bd85b None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:15.502113 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4ba514-1175-40ce-a760-3fe5209bd85b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:16:15.502444 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4ba514-1175-40ce-a760-3fe5209bd85b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:15.504388 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4ba514-1175-40ce-a760-3fe5209bd85b None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:15.505358 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54d872f-1254-4c0e-8554-bc8761b529ba None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:15.521914 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 537/1070] 10.222.0.33 () {56 vars in 1133 bytes} [Fri Jul 15 19:16:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:16:15.523037 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe980be6-b2d7-4ed7-a527-e61a41a20345 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:15.523326 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe980be6-b2d7-4ed7-a527-e61a41a20345 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:15.523606 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe980be6-b2d7-4ed7-a527-e61a41a20345 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:15.524861 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 538/1071] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:16:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:16:15.527363 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe980be6-b2d7-4ed7-a527-e61a41a20345 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:15.541030 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ea6f00-7210-4bdc-b60d-a91e25cb7276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:15.541756 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ea6f00-7210-4bdc-b60d-a91e25cb7276 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:15.542055 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ea6f00-7210-4bdc-b60d-a91e25cb7276 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:15.542328 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ea6f00-7210-4bdc-b60d-a91e25cb7276 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:15.543115 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ea6f00-7210-4bdc-b60d-a91e25cb7276 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:16:15.543419 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ea6f00-7210-4bdc-b60d-a91e25cb7276 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:15.544786 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ea6f00-7210-4bdc-b60d-a91e25cb7276 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:15.553315 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 534/1072] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 15 19:16:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:16:15.559136 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261148f0-5846-4d91-8cfd-721a1fd9d304 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:15.559431 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261148f0-5846-4d91-8cfd-721a1fd9d304 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:15.559740 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261148f0-5846-4d91-8cfd-721a1fd9d304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:15.575313 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-261148f0-5846-4d91-8cfd-721a1fd9d304 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:16:15.576498 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-261148f0-5846-4d91-8cfd-721a1fd9d304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:16:15.629930 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe980be6-b2d7-4ed7-a527-e61a41a20345 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:15.643577 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 535/1073] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:16:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:16:15.644406 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261148f0-5846-4d91-8cfd-721a1fd9d304 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:15.656313 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:15.664727 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4ccc4a-3fc3-41ab-aefb-3115bd8d0058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:15.667988 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4ccc4a-3fc3-41ab-aefb-3115bd8d0058 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:15.667988 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4ccc4a-3fc3-41ab-aefb-3115bd8d0058 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:15.667988 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4ccc4a-3fc3-41ab-aefb-3115bd8d0058 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:15.676561 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4ccc4a-3fc3-41ab-aefb-3115bd8d0058 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:15.678088 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4ccc4a-3fc3-41ab-aefb-3115bd8d0058 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:15.697410 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:15.704367 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-bd4ccc4a-3fc3-41ab-aefb-3115bd8d0058 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:16:15.704367 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:16:15.704367 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:16:15.704367 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:16:15.704367 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:16:15.704367 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:16:15.704367 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:16:15.704367 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:16:15.704367 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:16:15.704367 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:16:15.704367 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:16:15.704367 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:16:15.704367 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:16:15.704367 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:16:15.704367 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:16:15.715272 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:16:15.715272 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:16:15.715272 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:16:15.715272 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:16:15.715272 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:16:15.715272 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:16:15.715272 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:16:15.715272 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:16:15.715272 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:16:15.715272 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:16:15.715272 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:16:15.715272 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:16:15.715272 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:16:15.715272 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:16:15.715272 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:16:15.715272 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:16:15.715272 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:16:15.716602 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:16:15.716602 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:16:15.716602 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:16:15.716602 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:16:15.716602 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 539/1074] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:16:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:16:15.716602 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4ccc4a-3fc3-41ab-aefb-3115bd8d0058 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:15.750252 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261148f0-5846-4d91-8cfd-721a1fd9d304 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:15.787416 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927a2deb-0354-4095-bfb8-42469c8bd039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:15.788279 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a2deb-0354-4095-bfb8-42469c8bd039 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:15.788555 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a2deb-0354-4095-bfb8-42469c8bd039 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:15.788820 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a2deb-0354-4095-bfb8-42469c8bd039 None admin] PATH_INFO: `/v3/users/5f76859b74f14fc499f2c658439e8129` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:15.790045 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927a2deb-0354-4095-bfb8-42469c8bd039 None admin] RBAC: Authorizing `identity:get_user(user_id=5f76859b74f14fc499f2c658439e8129)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:15.791465 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927a2deb-0354-4095-bfb8-42469c8bd039 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:15.793080 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 536/1075] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 15 19:16:15 2022] GET /identity/v3/users/5f76859b74f14fc499f2c658439e8129 => generated 285 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:16:15.794062 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927a2deb-0354-4095-bfb8-42469c8bd039 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:15.812078 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4ccc4a-3fc3-41ab-aefb-3115bd8d0058 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:15.846042 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6eccc4-6bf0-4023-842a-dba76b1dc19c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:15.847194 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6eccc4-6bf0-4023-842a-dba76b1dc19c None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:15.847476 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6eccc4-6bf0-4023-842a-dba76b1dc19c None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:15.847752 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6eccc4-6bf0-4023-842a-dba76b1dc19c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:15.895234 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:15.895406 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927a2deb-0354-4095-bfb8-42469c8bd039 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:15.914446 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6eccc4-6bf0-4023-842a-dba76b1dc19c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:15.915109 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:15.916043 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6eccc4-6bf0-4023-842a-dba76b1dc19c None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:15.928691 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db40145-3e0d-4bd5-9cba-31e9ead73e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:15.929351 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db40145-3e0d-4bd5-9cba-31e9ead73e07 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:15.929627 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db40145-3e0d-4bd5-9cba-31e9ead73e07 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:15.929969 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db40145-3e0d-4bd5-9cba-31e9ead73e07 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:15.930635 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db40145-3e0d-4bd5-9cba-31e9ead73e07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:16:15.930933 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db40145-3e0d-4bd5-9cba-31e9ead73e07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:15.932317 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db40145-3e0d-4bd5-9cba-31e9ead73e07 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:15.948155 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 537/1076] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:16:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:16:15.948155 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db40145-3e0d-4bd5-9cba-31e9ead73e07 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:16.020633 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 540/1077] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 15 19:16:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 19:16:16.021489 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6eccc4-6bf0-4023-842a-dba76b1dc19c None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:16.047583 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db40145-3e0d-4bd5-9cba-31e9ead73e07 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:16.081682 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faee1da8-c94b-409a-93bf-8fc61290e21d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:16.082589 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee1da8-c94b-409a-93bf-8fc61290e21d None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:16.082878 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee1da8-c94b-409a-93bf-8fc61290e21d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:16.083124 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee1da8-c94b-409a-93bf-8fc61290e21d None admin] PATH_INFO: `/v3/projects/ab5433faa0b14ec29f68a8b8219b16ed` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:16.084296 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faee1da8-c94b-409a-93bf-8fc61290e21d None admin] RBAC: Authorizing `identity:get_project(project_id=ab5433faa0b14ec29f68a8b8219b16ed)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:16.085717 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faee1da8-c94b-409a-93bf-8fc61290e21d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:16.087573 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 538/1078] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 15 19:16:15 2022] GET /identity/v3/projects/ab5433faa0b14ec29f68a8b8219b16ed => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:16:16.088170 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faee1da8-c94b-409a-93bf-8fc61290e21d None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:16.101435 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:16.120579 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6eccc4-6bf0-4023-842a-dba76b1dc19c None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:16.136825 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:16.155339 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee8516d-9686-4e74-8043-4288fbbc8c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:16.156131 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee8516d-9686-4e74-8043-4288fbbc8c96 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:16.156418 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee8516d-9686-4e74-8043-4288fbbc8c96 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:16.156691 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee8516d-9686-4e74-8043-4288fbbc8c96 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:16.163472 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee8516d-9686-4e74-8043-4288fbbc8c96 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:16.164918 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee8516d-9686-4e74-8043-4288fbbc8c96 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:16.178870 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-7ee8516d-9686-4e74-8043-4288fbbc8c96 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:16:16.178870 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:16:16.178870 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:16:16.178870 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:16:16.178870 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:16:16.178870 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:16:16.178870 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:16:16.178870 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:16:16.178870 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:16:16.178870 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:16:16.178870 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:16:16.178870 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:16:16.178870 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 19:16:16.178870 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 19:16:16.178870 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 19:16:16.179962 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 19:16:16.179962 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:16:16.179962 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:16:16.179962 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:16:16.179962 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:16:16.179962 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:16:16.179962 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:16:16.179962 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:16:16.179962 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:16:16.179962 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:16:16.179962 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:16:16.179962 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:16:16.179962 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:16:16.179962 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:16:16.179962 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:16:16.179962 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:16:16.179962 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:16:16.181005 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 19:16:16.181005 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 19:16:16.181005 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:16:16.181005 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:16:16.181005 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:16:16.181005 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:16:16.181005 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:16:16.181005 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:16:16.181005 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 541/1079] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 15 19:16:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:16:16.181005 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee8516d-9686-4e74-8043-4288fbbc8c96 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:16.189367 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faee1da8-c94b-409a-93bf-8fc61290e21d None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:16.222944 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd631c70-cfe8-4489-902e-11132f39b01c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:16.223640 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd631c70-cfe8-4489-902e-11132f39b01c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:16.223925 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd631c70-cfe8-4489-902e-11132f39b01c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:16.224204 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd631c70-cfe8-4489-902e-11132f39b01c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:16.235323 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd631c70-cfe8-4489-902e-11132f39b01c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:16.237240 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd631c70-cfe8-4489-902e-11132f39b01c None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:16.254241 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-fd631c70-cfe8-4489-902e-11132f39b01c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:16:16.254241 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:16:16.254241 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:16:16.254241 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:16:16.254241 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:16:16.254241 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:16:16.254241 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:16:16.254241 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:16:16.254241 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:16:16.254241 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:16:16.254241 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:16:16.254241 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:16:16.254241 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:16:16.254241 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:16:16.254241 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:16:16.255230 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:16:16.255230 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:16:16.255230 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:16:16.255230 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:16:16.255230 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:16:16.255230 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:16:16.255230 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:16:16.255230 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:16:16.255230 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:16:16.255230 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:16:16.255230 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:16:16.255230 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:16:16.255230 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:16:16.255230 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:16:16.255230 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:16:16.255230 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:16:16.255230 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:16:16.261433 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:16:16.261433 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:16:16.261433 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:16:16.261433 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:16:16.261433 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 539/1080] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:16:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:16:16.261433 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd631c70-cfe8-4489-902e-11132f39b01c None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:16.288300 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee8516d-9686-4e74-8043-4288fbbc8c96 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:16.321876 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc596adc-0715-479d-b7ad-b498381f798a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:16.322758 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc596adc-0715-479d-b7ad-b498381f798a None admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:16.323103 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc596adc-0715-479d-b7ad-b498381f798a None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:16.323298 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc596adc-0715-479d-b7ad-b498381f798a None admin] PATH_INFO: `/v3/projects/ab5433faa0b14ec29f68a8b8219b16ed/users/5f76859b74f14fc499f2c658439e8129/roles/4318f86ba9f74cd4a0bef1c311276657` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:16.325401 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc596adc-0715-479d-b7ad-b498381f798a None admin] RBAC: Authorizing `identity:create_grant(project_id=ab5433faa0b14ec29f68a8b8219b16ed, user_id=5f76859b74f14fc499f2c658439e8129, role_id=4318f86ba9f74cd4a0bef1c311276657)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:16.326885 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc596adc-0715-479d-b7ad-b498381f798a None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:16.354703 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:16.357649 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd631c70-cfe8-4489-902e-11132f39b01c None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:16.420234 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:16.425069 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 542/1081] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jul 15 19:16:16 2022] PUT /identity/v3/projects/ab5433faa0b14ec29f68a8b8219b16ed/users/5f76859b74f14fc499f2c658439e8129/roles/4318f86ba9f74cd4a0bef1c311276657 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:16:16.425890 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc596adc-0715-479d-b7ad-b498381f798a None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:16.452688 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68622e0-a8d6-4a77-8008-000aebf5d9a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:16.453381 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68622e0-a8d6-4a77-8008-000aebf5d9a4 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:16.453665 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68622e0-a8d6-4a77-8008-000aebf5d9a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:16.453973 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68622e0-a8d6-4a77-8008-000aebf5d9a4 None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:16.454785 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68622e0-a8d6-4a77-8008-000aebf5d9a4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:16:16.455098 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68622e0-a8d6-4a77-8008-000aebf5d9a4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:16.457171 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68622e0-a8d6-4a77-8008-000aebf5d9a4 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:16.471058 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 540/1082] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jul 15 19:16:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:16:16.478527 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68622e0-a8d6-4a77-8008-000aebf5d9a4 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:16.551544 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc596adc-0715-479d-b7ad-b498381f798a None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:16.590266 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892e2001-e1cb-4522-8825-49d9f32ea568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:16.590949 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892e2001-e1cb-4522-8825-49d9f32ea568 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:16.591230 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892e2001-e1cb-4522-8825-49d9f32ea568 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:16.591506 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892e2001-e1cb-4522-8825-49d9f32ea568 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:16.592234 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892e2001-e1cb-4522-8825-49d9f32ea568 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:16:16.592535 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892e2001-e1cb-4522-8825-49d9f32ea568 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:16.595393 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892e2001-e1cb-4522-8825-49d9f32ea568 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:16.600702 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68622e0-a8d6-4a77-8008-000aebf5d9a4 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:16.607202 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 543/1083] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:16:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:16:16.639494 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c25596a-1d63-461f-8505-69fb751617e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:16.640489 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c25596a-1d63-461f-8505-69fb751617e0 None admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:16.640780 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c25596a-1d63-461f-8505-69fb751617e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:16.641049 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c25596a-1d63-461f-8505-69fb751617e0 None admin] PATH_INFO: `/v3/projects/55a756ccaef14cb49b26b0d68dd0b466/users/7673a3b5c0324e07a9535ac70ed7a491/roles/c1dc7198be2d43d3aa44d16107d9a959` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:16.644083 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c25596a-1d63-461f-8505-69fb751617e0 None admin] RBAC: Authorizing `identity:create_grant(project_id=55a756ccaef14cb49b26b0d68dd0b466, user_id=7673a3b5c0324e07a9535ac70ed7a491, role_id=c1dc7198be2d43d3aa44d16107d9a959)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:16.645846 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c25596a-1d63-461f-8505-69fb751617e0 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:16.717508 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 541/1084] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jul 15 19:16:16 2022] PUT /identity/v3/projects/55a756ccaef14cb49b26b0d68dd0b466/users/7673a3b5c0324e07a9535ac70ed7a491/roles/c1dc7198be2d43d3aa44d16107d9a959 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:16:18.421038 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91440ed-870b-48c3-9d3c-b809af52709b None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:18.421603 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91440ed-870b-48c3-9d3c-b809af52709b None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:18.421603 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91440ed-870b-48c3-9d3c-b809af52709b None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:18.422974 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 544/1085] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:16:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:16:18.428629 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e3d69e-b399-4b0e-a13c-6222b129d768 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:18.429048 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e3d69e-b399-4b0e-a13c-6222b129d768 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:18.429191 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e3d69e-b399-4b0e-a13c-6222b129d768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:18.446319 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-54e3d69e-b399-4b0e-a13c-6222b129d768 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:16:18.447513 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-54e3d69e-b399-4b0e-a13c-6222b129d768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:16:18.461344 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:18.513368 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 542/1086] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:16:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:16:18.537245 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91440ed-870b-48c3-9d3c-b809af52709b None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:18.597012 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:18.637557 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91440ed-870b-48c3-9d3c-b809af52709b None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:18.671792 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cc1c3f-44d3-4b53-8f6d-5b6ed233b756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:18.672516 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc1c3f-44d3-4b53-8f6d-5b6ed233b756 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:18.672796 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc1c3f-44d3-4b53-8f6d-5b6ed233b756 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:18.673066 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc1c3f-44d3-4b53-8f6d-5b6ed233b756 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:18.683117 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cc1c3f-44d3-4b53-8f6d-5b6ed233b756 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:16:18.691243 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-b6cc1c3f-44d3-4b53-8f6d-5b6ed233b756 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:16:18.691243 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:16:18.691243 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:16:18.691243 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:16:18.691243 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:16:18.691243 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:16:18.691243 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:16:18.691243 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:16:18.691243 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:16:18.691243 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:16:18.691243 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:16:18.691243 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:16:18.691243 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:16:18.691243 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:16:18.691243 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:16:18.691243 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:16:18.692350 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:16:18.692350 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:16:18.692350 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:16:18.692350 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:16:18.692350 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:16:18.692350 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:16:18.692350 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:16:18.692350 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:16:18.692350 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:16:18.692350 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:16:18.692350 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:16:18.692350 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:16:18.692350 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:16:18.692350 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:16:18.692350 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:16:18.692350 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:16:18.693203 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:16:18.693203 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:16:18.693203 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:16:18.693203 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:16:18.693203 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:16:18.693203 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:16:18.693203 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:16:18.693203 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:16:18.693203 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:16:18.693203 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:16:18.693203 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:16:18.693203 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:16:18.693203 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 545/1087] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jul 15 19:16:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:16:18.698488 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e3d69e-b399-4b0e-a13c-6222b129d768 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:18.710502 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1908ecf-85aa-49b5-a5da-6e806b8cb85b None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:18.710790 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1908ecf-85aa-49b5-a5da-6e806b8cb85b None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:18.711067 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1908ecf-85aa-49b5-a5da-6e806b8cb85b None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:18.712429 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 546/1088] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:16:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 19:16:18.717239 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94875524-aadd-49de-9c4b-16ab64c78cda None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:18.717524 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94875524-aadd-49de-9c4b-16ab64c78cda None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:18.717829 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94875524-aadd-49de-9c4b-16ab64c78cda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:18.733913 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-94875524-aadd-49de-9c4b-16ab64c78cda None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:16:18.735107 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-94875524-aadd-49de-9c4b-16ab64c78cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:16:18.797474 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e3d69e-b399-4b0e-a13c-6222b129d768 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:18.801544 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 547/1089] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:16:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 19:16:18.824850 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94875524-aadd-49de-9c4b-16ab64c78cda None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:18.834379 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73df36a-a267-4629-90a6-0f87c3e4a172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:18.835042 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73df36a-a267-4629-90a6-0f87c3e4a172 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:18.835321 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73df36a-a267-4629-90a6-0f87c3e4a172 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:18.835591 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73df36a-a267-4629-90a6-0f87c3e4a172 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:18.836344 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73df36a-a267-4629-90a6-0f87c3e4a172 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:16:18.836643 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73df36a-a267-4629-90a6-0f87c3e4a172 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:18.838059 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73df36a-a267-4629-90a6-0f87c3e4a172 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:18.846789 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 543/1090] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 15 19:16:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:16:18.852477 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73df36a-a267-4629-90a6-0f87c3e4a172 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:18.866795 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:18.874584 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:18.924089 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94875524-aadd-49de-9c4b-16ab64c78cda None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:18.951962 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73df36a-a267-4629-90a6-0f87c3e4a172 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:18.957869 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b56d192-050c-4faf-bef8-1fead8326dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:18.958681 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56d192-050c-4faf-bef8-1fead8326dc8 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:18.958964 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56d192-050c-4faf-bef8-1fead8326dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:18.959241 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56d192-050c-4faf-bef8-1fead8326dc8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:18.969538 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b56d192-050c-4faf-bef8-1fead8326dc8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:16:18.978232 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-7b56d192-050c-4faf-bef8-1fead8326dc8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 15 19:16:18.978232 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:16:18.978232 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:16:18.978232 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:16:18.978232 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:16:18.978232 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:16:18.978232 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:16:18.978232 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:16:18.978232 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:16:18.978232 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:16:18.978232 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:16:18.978232 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:16:18.978232 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:16:18.978232 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:16:18.978232 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:16:18.978232 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:16:18.979187 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:16:18.979187 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:16:18.979187 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:16:18.979187 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:16:18.979187 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:16:18.979187 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:16:18.979187 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:16:18.979187 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:16:18.979187 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:16:18.979187 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:16:18.979187 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:16:18.979187 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:16:18.979187 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:16:18.979187 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:16:18.979187 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:16:18.979187 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:16:18.980047 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:16:18.980047 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:16:18.980047 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:16:18.980047 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:16:18.980047 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:16:18.980047 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:16:18.980047 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:16:18.980047 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:16:18.980047 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:16:18.980047 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:16:18.980047 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 15 19:16:18.980047 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:16:18.980047 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 548/1091] 10.222.0.33 () {56 vars in 1120 bytes} [Fri Jul 15 19:16:18 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:16:18.985254 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b56d192-050c-4faf-bef8-1fead8326dc8 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:18.987601 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad10532-e23e-439c-95ac-18c77c06f6fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:18.988476 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad10532-e23e-439c-95ac-18c77c06f6fa None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:18.988760 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad10532-e23e-439c-95ac-18c77c06f6fa None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:18.989035 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad10532-e23e-439c-95ac-18c77c06f6fa None admin] PATH_INFO: `/v3/users/5f76859b74f14fc499f2c658439e8129` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:18.990267 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad10532-e23e-439c-95ac-18c77c06f6fa None admin] RBAC: Authorizing `identity:get_user(user_id=5f76859b74f14fc499f2c658439e8129)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:18.991699 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad10532-e23e-439c-95ac-18c77c06f6fa None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:18.998104 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 544/1092] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 15 19:16:18 2022] GET /identity/v3/users/5f76859b74f14fc499f2c658439e8129 => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:16:18.998923 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad10532-e23e-439c-95ac-18c77c06f6fa None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:19.077932 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:19.082952 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b56d192-050c-4faf-bef8-1fead8326dc8 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:19.093211 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:19.097681 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad10532-e23e-439c-95ac-18c77c06f6fa None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:19.117101 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f617d6-35e9-4c91-b756-3a38cdc6554b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:19.117781 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f617d6-35e9-4c91-b756-3a38cdc6554b None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:19.118069 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f617d6-35e9-4c91-b756-3a38cdc6554b None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:19.118349 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f617d6-35e9-4c91-b756-3a38cdc6554b None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:19.119094 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f617d6-35e9-4c91-b756-3a38cdc6554b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:16:19.121707 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f617d6-35e9-4c91-b756-3a38cdc6554b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:19.121707 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f617d6-35e9-4c91-b756-3a38cdc6554b None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:19.129225 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 549/1093] 10.222.0.33 () {56 vars in 1115 bytes} [Fri Jul 15 19:16:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:16:19.132081 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc63f23d-9a59-4ef6-93b1-c5b8d7a8e518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:19.132734 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc63f23d-9a59-4ef6-93b1-c5b8d7a8e518 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:19.133015 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc63f23d-9a59-4ef6-93b1-c5b8d7a8e518 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:19.133290 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc63f23d-9a59-4ef6-93b1-c5b8d7a8e518 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:19.134847 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f617d6-35e9-4c91-b756-3a38cdc6554b None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:19.202226 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc63f23d-9a59-4ef6-93b1-c5b8d7a8e518 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:19.203596 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc63f23d-9a59-4ef6-93b1-c5b8d7a8e518 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:19.232536 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f617d6-35e9-4c91-b756-3a38cdc6554b None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:19.266861 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12c6113-226d-4fd4-8d71-1aaedb083008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:19.267641 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12c6113-226d-4fd4-8d71-1aaedb083008 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:19.267954 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12c6113-226d-4fd4-8d71-1aaedb083008 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:19.268230 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12c6113-226d-4fd4-8d71-1aaedb083008 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:19.278469 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12c6113-226d-4fd4-8d71-1aaedb083008 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:19.279896 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12c6113-226d-4fd4-8d71-1aaedb083008 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:19.282008 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:19.297975 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-a12c6113-226d-4fd4-8d71-1aaedb083008 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:16:19.297975 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:16:19.297975 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:16:19.297975 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:16:19.297975 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:16:19.297975 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:16:19.297975 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:16:19.297975 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:16:19.297975 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:16:19.297975 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:16:19.297975 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:16:19.297975 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:16:19.297975 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:16:19.297975 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:16:19.297975 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:16:19.299118 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:16:19.299118 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:16:19.299118 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:16:19.299118 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:16:19.299118 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:16:19.299118 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:16:19.299118 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:16:19.299118 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:16:19.299118 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:16:19.299118 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:16:19.299118 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:16:19.299118 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:16:19.299118 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:16:19.299118 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:16:19.299118 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:16:19.299118 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:16:19.299118 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:16:19.300536 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:16:19.300536 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:16:19.300536 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:16:19.300536 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:16:19.300536 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 550/1094] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:16:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:16:19.309238 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 545/1095] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 15 19:16:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 15 19:16:19.309379 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12c6113-226d-4fd4-8d71-1aaedb083008 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:19.310261 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc63f23d-9a59-4ef6-93b1-c5b8d7a8e518 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:19.332506 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:19.410553 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12c6113-226d-4fd4-8d71-1aaedb083008 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:19.411306 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc63f23d-9a59-4ef6-93b1-c5b8d7a8e518 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:19.444741 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd95d97-b47a-4be7-bbec-f2a1bacbb31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:19.445264 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d7754b-e7bd-406b-b569-fa8e8a13f02a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:19.445652 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd95d97-b47a-4be7-bbec-f2a1bacbb31f None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:19.445652 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd95d97-b47a-4be7-bbec-f2a1bacbb31f None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:19.446128 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd95d97-b47a-4be7-bbec-f2a1bacbb31f None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:19.446231 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d7754b-e7bd-406b-b569-fa8e8a13f02a None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:19.446307 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d7754b-e7bd-406b-b569-fa8e8a13f02a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:19.446532 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d7754b-e7bd-406b-b569-fa8e8a13f02a None admin] PATH_INFO: `/v3/projects/ab5433faa0b14ec29f68a8b8219b16ed` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:19.446690 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd95d97-b47a-4be7-bbec-f2a1bacbb31f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:16:19.446987 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd95d97-b47a-4be7-bbec-f2a1bacbb31f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:19.447857 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d7754b-e7bd-406b-b569-fa8e8a13f02a None admin] RBAC: Authorizing `identity:get_project(project_id=ab5433faa0b14ec29f68a8b8219b16ed)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:19.448351 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd95d97-b47a-4be7-bbec-f2a1bacbb31f None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:19.449278 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d7754b-e7bd-406b-b569-fa8e8a13f02a None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:19.451274 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 546/1096] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 15 19:16:19 2022] GET /identity/v3/projects/ab5433faa0b14ec29f68a8b8219b16ed => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:16:19.457138 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d7754b-e7bd-406b-b569-fa8e8a13f02a None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:19.464664 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 551/1097] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:16:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:16:19.465502 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd95d97-b47a-4be7-bbec-f2a1bacbb31f None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:19.507279 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:19.550099 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:19.558086 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d7754b-e7bd-406b-b569-fa8e8a13f02a None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:19.566260 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd95d97-b47a-4be7-bbec-f2a1bacbb31f None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:19.593421 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729096b4-e3bb-4e42-a8ed-10d610e1ae7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:19.594147 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729096b4-e3bb-4e42-a8ed-10d610e1ae7c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:19.594432 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729096b4-e3bb-4e42-a8ed-10d610e1ae7c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:19.594708 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729096b4-e3bb-4e42-a8ed-10d610e1ae7c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:19.595849 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729096b4-e3bb-4e42-a8ed-10d610e1ae7c None admin] RBAC: Adding query filter params (role.id=4318f86ba9f74cd4a0bef1c311276657, scope.project.id=ab5433faa0b14ec29f68a8b8219b16ed, user.id=5f76859b74f14fc499f2c658439e8129) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:16:19.595849 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729096b4-e3bb-4e42-a8ed-10d610e1ae7c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:19.597421 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729096b4-e3bb-4e42-a8ed-10d610e1ae7c None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:19.600094 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc8114c-e35f-48af-afa9-3b116ac6c833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:19.600901 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc8114c-e35f-48af-afa9-3b116ac6c833 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:19.601183 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc8114c-e35f-48af-afa9-3b116ac6c833 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:19.601456 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc8114c-e35f-48af-afa9-3b116ac6c833 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:19.603451 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 547/1098] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 15 19:16:19 2022] GET /identity/v3/role_assignments?user.id=5f76859b74f14fc499f2c658439e8129&scope.project.id=ab5433faa0b14ec29f68a8b8219b16ed&role.id=4318f86ba9f74cd4a0bef1c311276657 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:16:19.609254 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc8114c-e35f-48af-afa9-3b116ac6c833 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:19.610852 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc8114c-e35f-48af-afa9-3b116ac6c833 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:19.627272 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-bdc8114c-e35f-48af-afa9-3b116ac6c833 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 15 19:16:19.627272 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:16:19.627272 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:16:19.627272 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:16:19.627272 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:16:19.627272 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:16:19.627272 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:16:19.627272 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:16:19.627272 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:16:19.627272 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:16:19.627272 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:16:19.627272 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:16:19.627272 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 19:16:19.627272 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 19:16:19.627272 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 19:16:19.627272 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 19:16:19.628909 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:16:19.628909 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:16:19.628909 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:16:19.628909 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:16:19.628909 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:16:19.628909 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:16:19.628909 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:16:19.628909 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:16:19.628909 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:16:19.628909 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:16:19.628909 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:16:19.628909 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:16:19.628909 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:16:19.628909 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:16:19.628909 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:16:19.628909 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:16:19.628909 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:16:19.628909 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:16:19.629908 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:16:19.629908 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:16:19.629908 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 19:16:19.629908 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 19:16:19.629908 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 19:16:19.629908 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 19:16:19.629908 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 19:16:19.629908 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 19:16:19.629908 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 15 19:16:19.629908 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:16:19.633992 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 552/1099] 10.222.0.33 () {56 vars in 1093 bytes} [Fri Jul 15 19:16:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:16:19.635246 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729096b4-e3bb-4e42-a8ed-10d610e1ae7c None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:19.751058 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729096b4-e3bb-4e42-a8ed-10d610e1ae7c None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:19.766170 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:19.787885 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12300b7b-3cd6-4d9f-a934-05600079c3ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:19.788556 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12300b7b-3cd6-4d9f-a934-05600079c3ef None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:19.788842 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12300b7b-3cd6-4d9f-a934-05600079c3ef None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:19.789116 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12300b7b-3cd6-4d9f-a934-05600079c3ef None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:19.790015 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12300b7b-3cd6-4d9f-a934-05600079c3ef None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:16:19.790323 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12300b7b-3cd6-4d9f-a934-05600079c3ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:19.791754 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12300b7b-3cd6-4d9f-a934-05600079c3ef None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:19.814509 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 548/1100] 10.222.0.33 () {56 vars in 1133 bytes} [Fri Jul 15 19:16:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:16:19.817060 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc8114c-e35f-48af-afa9-3b116ac6c833 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:19.914995 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc8114c-e35f-48af-afa9-3b116ac6c833 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:19.949142 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db3b503-d2a5-4c15-b095-0b8959e8c9f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:19.949813 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db3b503-d2a5-4c15-b095-0b8959e8c9f9 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:19.950100 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db3b503-d2a5-4c15-b095-0b8959e8c9f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:19.950379 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db3b503-d2a5-4c15-b095-0b8959e8c9f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:19.964721 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:20.018544 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db3b503-d2a5-4c15-b095-0b8959e8c9f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:20.020161 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db3b503-d2a5-4c15-b095-0b8959e8c9f9 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:20.121276 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 553/1101] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 15 19:16:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 19:16:20.122556 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12300b7b-3cd6-4d9f-a934-05600079c3ef None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:20.220378 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12300b7b-3cd6-4d9f-a934-05600079c3ef None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:20.254409 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e571fd9a-71a0-427f-9f1f-3d7cd5942b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:20.255201 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e571fd9a-71a0-427f-9f1f-3d7cd5942b74 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:20.255484 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e571fd9a-71a0-427f-9f1f-3d7cd5942b74 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:20.255759 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e571fd9a-71a0-427f-9f1f-3d7cd5942b74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:20.265850 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e571fd9a-71a0-427f-9f1f-3d7cd5942b74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:20.267290 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e571fd9a-71a0-427f-9f1f-3d7cd5942b74 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:20.285269 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-e571fd9a-71a0-427f-9f1f-3d7cd5942b74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:16:20.285269 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:16:20.285269 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:16:20.285269 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:16:20.285269 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:16:20.285269 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:16:20.285269 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:16:20.285269 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:16:20.285269 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:16:20.285269 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:16:20.285269 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:16:20.285269 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:16:20.285269 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:16:20.285269 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:16:20.285269 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:16:20.290943 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:16:20.290943 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:16:20.290943 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:16:20.290943 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:16:20.290943 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:16:20.290943 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:16:20.290943 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:16:20.290943 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:16:20.290943 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:16:20.290943 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:16:20.290943 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:16:20.290943 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:16:20.290943 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:16:20.290943 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:16:20.290943 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:16:20.290943 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:16:20.290943 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:16:20.292036 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:16:20.292036 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:16:20.292036 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:16:20.292036 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:16:20.292036 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 549/1102] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:16:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:16:20.292357 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db3b503-d2a5-4c15-b095-0b8959e8c9f9 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:20.338380 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:20.392714 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db3b503-d2a5-4c15-b095-0b8959e8c9f9 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:20.433550 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc6422b-338f-4438-8d09-9f098868f053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:20.434647 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc6422b-338f-4438-8d09-9f098868f053 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:20.434647 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc6422b-338f-4438-8d09-9f098868f053 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:20.434867 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc6422b-338f-4438-8d09-9f098868f053 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:20.435490 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc6422b-338f-4438-8d09-9f098868f053 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:16:20.435796 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc6422b-338f-4438-8d09-9f098868f053 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:20.437150 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc6422b-338f-4438-8d09-9f098868f053 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:20.449411 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 554/1103] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:16:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:16:20.455845 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e571fd9a-71a0-427f-9f1f-3d7cd5942b74 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:20.479861 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:20.553262 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e571fd9a-71a0-427f-9f1f-3d7cd5942b74 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:20.585660 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a051419-2030-4cba-a91f-86bab263457d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:20.586460 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a051419-2030-4cba-a91f-86bab263457d None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:20.586734 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a051419-2030-4cba-a91f-86bab263457d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:20.587003 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a051419-2030-4cba-a91f-86bab263457d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:20.593431 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a051419-2030-4cba-a91f-86bab263457d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:20.594892 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a051419-2030-4cba-a91f-86bab263457d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:20.609018 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-5a051419-2030-4cba-a91f-86bab263457d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:16:20.609018 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:16:20.609018 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:16:20.609018 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:16:20.609018 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:16:20.609018 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:16:20.609018 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:16:20.609018 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:16:20.609018 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:16:20.609018 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:16:20.609018 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:16:20.609018 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:16:20.609018 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 19:16:20.609018 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 19:16:20.609018 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 19:16:20.610075 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 19:16:20.610075 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:16:20.610075 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:16:20.610075 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:16:20.610075 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:16:20.610075 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:16:20.610075 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:16:20.610075 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:16:20.610075 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:16:20.610075 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:16:20.610075 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:16:20.610075 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:16:20.610075 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:16:20.610075 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:16:20.610075 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:16:20.610075 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:16:20.610075 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:16:20.610999 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 19:16:20.610999 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 19:16:20.610999 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:16:20.610999 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:16:20.610999 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:16:20.610999 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:16:20.610999 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:16:20.610999 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:16:20.610999 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 550/1104] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 15 19:16:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:16:20.616004 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc6422b-338f-4438-8d09-9f098868f053 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:20.712791 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc6422b-338f-4438-8d09-9f098868f053 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:20.715114 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:20.746239 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef65a4c-82eb-4e63-a53d-c25a940f3f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:20.746913 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef65a4c-82eb-4e63-a53d-c25a940f3f46 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:20.747196 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef65a4c-82eb-4e63-a53d-c25a940f3f46 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:20.747471 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef65a4c-82eb-4e63-a53d-c25a940f3f46 None admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:20.748284 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef65a4c-82eb-4e63-a53d-c25a940f3f46 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:16:20.748594 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef65a4c-82eb-4e63-a53d-c25a940f3f46 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:20.750236 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef65a4c-82eb-4e63-a53d-c25a940f3f46 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:20.766106 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 555/1105] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jul 15 19:16:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:16:20.774591 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a051419-2030-4cba-a91f-86bab263457d None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:20.859687 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:20.877788 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a051419-2030-4cba-a91f-86bab263457d None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:20.910958 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96ab5dc-687c-4aa0-bbbd-814088f9f4f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:20.911599 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96ab5dc-687c-4aa0-bbbd-814088f9f4f7 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:20.911878 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96ab5dc-687c-4aa0-bbbd-814088f9f4f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:20.912148 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96ab5dc-687c-4aa0-bbbd-814088f9f4f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:20.912968 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96ab5dc-687c-4aa0-bbbd-814088f9f4f7 None admin] RBAC: Adding query filter params (role.id=c1dc7198be2d43d3aa44d16107d9a959, scope.project.id=55a756ccaef14cb49b26b0d68dd0b466, user.id=7673a3b5c0324e07a9535ac70ed7a491) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:16:20.913273 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96ab5dc-687c-4aa0-bbbd-814088f9f4f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:20.914708 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96ab5dc-687c-4aa0-bbbd-814088f9f4f7 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:20.920371 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 551/1106] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 15 19:16:20 2022] GET /identity/v3/role_assignments?user.id=7673a3b5c0324e07a9535ac70ed7a491&scope.project.id=55a756ccaef14cb49b26b0d68dd0b466&role.id=c1dc7198be2d43d3aa44d16107d9a959 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:16:21.702908 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c19c21-6d31-4e60-bd2d-6b6cc66d6d86 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:21.702908 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c19c21-6d31-4e60-bd2d-6b6cc66d6d86 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:21.702908 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c19c21-6d31-4e60-bd2d-6b6cc66d6d86 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:21.702908 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 556/1107] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:16:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:16:21.703992 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92439a99-6088-4c08-8dc7-3b15d64f7660 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:21.703992 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92439a99-6088-4c08-8dc7-3b15d64f7660 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:21.703992 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92439a99-6088-4c08-8dc7-3b15d64f7660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:21.710679 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-92439a99-6088-4c08-8dc7-3b15d64f7660 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:16:21.711880 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-92439a99-6088-4c08-8dc7-3b15d64f7660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:16:21.788252 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 552/1108] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:16:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 19:16:21.812035 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c19c21-6d31-4e60-bd2d-6b6cc66d6d86 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:21.910227 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c19c21-6d31-4e60-bd2d-6b6cc66d6d86 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:21.943357 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e6f5f1-a1d1-4186-9f57-2758e903ae0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:21.944150 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6f5f1-a1d1-4186-9f57-2758e903ae0b None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:21.944426 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6f5f1-a1d1-4186-9f57-2758e903ae0b None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:21.944698 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6f5f1-a1d1-4186-9f57-2758e903ae0b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:21.945887 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e6f5f1-a1d1-4186-9f57-2758e903ae0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:21.947290 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e6f5f1-a1d1-4186-9f57-2758e903ae0b None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:21.948871 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 557/1109] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:16:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:16:21.954853 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92439a99-6088-4c08-8dc7-3b15d64f7660 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:22.029144 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:22.051581 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92439a99-6088-4c08-8dc7-3b15d64f7660 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:22.084094 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad7959d-cf1c-4c52-a06b-89ea154951f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:22.084749 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad7959d-cf1c-4c52-a06b-89ea154951f4 None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:22.085030 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad7959d-cf1c-4c52-a06b-89ea154951f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:22.085302 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad7959d-cf1c-4c52-a06b-89ea154951f4 None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:22.086028 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad7959d-cf1c-4c52-a06b-89ea154951f4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:22.087452 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad7959d-cf1c-4c52-a06b-89ea154951f4 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:22.155364 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 553/1110] 10.222.0.33 () {60 vars in 1132 bytes} [Fri Jul 15 19:16:21 2022] POST /identity/v3/users => generated 286 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:16:24.203309 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86e7033-37f1-4dfa-b642-4d9e60f500d7 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:24.203832 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86e7033-37f1-4dfa-b642-4d9e60f500d7 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:24.203832 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86e7033-37f1-4dfa-b642-4d9e60f500d7 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:24.205658 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 558/1111] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:16:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:16:24.210867 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54154d51-528d-4a8d-9984-2829debb8f8d None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:24.211251 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54154d51-528d-4a8d-9984-2829debb8f8d None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:24.211418 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54154d51-528d-4a8d-9984-2829debb8f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:24.227771 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-54154d51-528d-4a8d-9984-2829debb8f8d None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:16:24.228947 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-54154d51-528d-4a8d-9984-2829debb8f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:16:24.293355 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 554/1112] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:16:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:16:24.316851 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86e7033-37f1-4dfa-b642-4d9e60f500d7 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:24.348134 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:24.418335 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86e7033-37f1-4dfa-b642-4d9e60f500d7 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:24.454307 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61e4e9f-2884-46bf-b9a9-b97d779c3ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:24.455604 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61e4e9f-2884-46bf-b9a9-b97d779c3ff0 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:24.455889 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61e4e9f-2884-46bf-b9a9-b97d779c3ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:24.456164 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61e4e9f-2884-46bf-b9a9-b97d779c3ff0 None admin] PATH_INFO: `/v3/roles/e97d26621445466d9550261c608db9c5` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:24.457720 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61e4e9f-2884-46bf-b9a9-b97d779c3ff0 None admin] RBAC: Authorizing `identity:get_role(role_id=e97d26621445466d9550261c608db9c5)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:24.459107 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61e4e9f-2884-46bf-b9a9-b97d779c3ff0 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:24.460487 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 559/1113] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 15 19:16:24 2022] GET /identity/v3/roles/e97d26621445466d9550261c608db9c5 => generated 223 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:16:24.466807 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54154d51-528d-4a8d-9984-2829debb8f8d None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:24.526671 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:24.563632 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54154d51-528d-4a8d-9984-2829debb8f8d None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:24.694127 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2bb064-f03c-4a33-8390-549549f07cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:24.694991 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2bb064-f03c-4a33-8390-549549f07cfa None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:24.695273 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2bb064-f03c-4a33-8390-549549f07cfa None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:24.695548 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2bb064-f03c-4a33-8390-549549f07cfa None admin] PATH_INFO: `/v3/users/a9167ddd61e64e2fa7f3a29a69282148` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:24.708899 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2bb064-f03c-4a33-8390-549549f07cfa None admin] RBAC: Authorizing `identity:get_user(user_id=a9167ddd61e64e2fa7f3a29a69282148)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:24.710308 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2bb064-f03c-4a33-8390-549549f07cfa None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:24.711963 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 555/1114] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 15 19:16:24 2022] GET /identity/v3/users/a9167ddd61e64e2fa7f3a29a69282148 => generated 286 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:16:24.717787 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61e4e9f-2884-46bf-b9a9-b97d779c3ff0 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:24.814164 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:24.817098 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61e4e9f-2884-46bf-b9a9-b97d779c3ff0 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:24.849834 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531b0492-6a60-42bc-8df2-32564fa89b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:24.850482 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b0492-6a60-42bc-8df2-32564fa89b4b None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:24.850767 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b0492-6a60-42bc-8df2-32564fa89b4b None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:24.851043 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b0492-6a60-42bc-8df2-32564fa89b4b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:24.916551 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531b0492-6a60-42bc-8df2-32564fa89b4b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:24.918018 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531b0492-6a60-42bc-8df2-32564fa89b4b None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:25.010565 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:25.014072 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 560/1115] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 15 19:16:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 19:16:25.020761 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2bb064-f03c-4a33-8390-549549f07cfa None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:25.119456 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2bb064-f03c-4a33-8390-549549f07cfa None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:25.138438 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:25.152304 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e87832-78d7-4717-ba51-559940673c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:25.153825 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e87832-78d7-4717-ba51-559940673c58 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:25.153825 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e87832-78d7-4717-ba51-559940673c58 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:25.153825 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e87832-78d7-4717-ba51-559940673c58 None admin] PATH_INFO: `/v3/projects/ab5433faa0b14ec29f68a8b8219b16ed` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:25.154837 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e87832-78d7-4717-ba51-559940673c58 None admin] RBAC: Authorizing `identity:get_project(project_id=ab5433faa0b14ec29f68a8b8219b16ed)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:25.156264 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e87832-78d7-4717-ba51-559940673c58 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:25.158002 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 556/1116] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 15 19:16:25 2022] GET /identity/v3/projects/ab5433faa0b14ec29f68a8b8219b16ed => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:16:25.164785 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531b0492-6a60-42bc-8df2-32564fa89b4b None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:25.259708 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531b0492-6a60-42bc-8df2-32564fa89b4b None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:25.292611 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b749c3-8ad7-4002-9fde-12f17264f9cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:25.293187 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b749c3-8ad7-4002-9fde-12f17264f9cb None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:25.293461 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b749c3-8ad7-4002-9fde-12f17264f9cb None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:25.293728 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b749c3-8ad7-4002-9fde-12f17264f9cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:25.294566 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b749c3-8ad7-4002-9fde-12f17264f9cb None admin] RBAC: Adding query filter params (role.id=e97d26621445466d9550261c608db9c5, scope.project.id=ab5433faa0b14ec29f68a8b8219b16ed, user.id=a9167ddd61e64e2fa7f3a29a69282148) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:16:25.294873 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b749c3-8ad7-4002-9fde-12f17264f9cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:25.296265 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b749c3-8ad7-4002-9fde-12f17264f9cb None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:25.301529 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 561/1117] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 15 19:16:25 2022] GET /identity/v3/role_assignments?user.id=a9167ddd61e64e2fa7f3a29a69282148&scope.project.id=ab5433faa0b14ec29f68a8b8219b16ed&role.id=e97d26621445466d9550261c608db9c5 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:16:27.271314 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a8ce48-c25e-43f9-bb0e-5c4074785476 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:27.271833 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a8ce48-c25e-43f9-bb0e-5c4074785476 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:27.271833 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a8ce48-c25e-43f9-bb0e-5c4074785476 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:27.273161 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 557/1118] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:16:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:16:27.278300 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7531b0d-c7fc-466a-a80a-c8cf1f7755d7 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:27.278697 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7531b0d-c7fc-466a-a80a-c8cf1f7755d7 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:27.278853 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7531b0d-c7fc-466a-a80a-c8cf1f7755d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:27.295669 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-f7531b0d-c7fc-466a-a80a-c8cf1f7755d7 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:16:27.296849 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-f7531b0d-c7fc-466a-a80a-c8cf1f7755d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:16:27.353218 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:27.361223 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 562/1119] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:16:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:16:27.382893 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a8ce48-c25e-43f9-bb0e-5c4074785476 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:27.480599 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a8ce48-c25e-43f9-bb0e-5c4074785476 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:27.513418 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eba20c6-9975-4b2a-b387-89356ddad16f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:27.514289 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba20c6-9975-4b2a-b387-89356ddad16f None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:27.514568 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba20c6-9975-4b2a-b387-89356ddad16f None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:27.514840 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba20c6-9975-4b2a-b387-89356ddad16f None admin] PATH_INFO: `/v3/roles/e97d26621445466d9550261c608db9c5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:27.516298 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eba20c6-9975-4b2a-b387-89356ddad16f None admin] RBAC: Authorizing `identity:get_role(role_id=e97d26621445466d9550261c608db9c5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:27.518340 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eba20c6-9975-4b2a-b387-89356ddad16f None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:27.520195 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 558/1120] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 15 19:16:27 2022] GET /identity/v3/roles/e97d26621445466d9550261c608db9c5 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:16:27.526330 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7531b0d-c7fc-466a-a80a-c8cf1f7755d7 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:27.620785 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7531b0d-c7fc-466a-a80a-c8cf1f7755d7 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:27.654188 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ada0942-e93c-44b9-b831-d5fcc78402b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:27.655050 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ada0942-e93c-44b9-b831-d5fcc78402b7 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:27.655328 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ada0942-e93c-44b9-b831-d5fcc78402b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:27.655598 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ada0942-e93c-44b9-b831-d5fcc78402b7 None admin] PATH_INFO: `/v3/users/a9167ddd61e64e2fa7f3a29a69282148` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:27.656771 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ada0942-e93c-44b9-b831-d5fcc78402b7 None admin] RBAC: Authorizing `identity:get_user(user_id=a9167ddd61e64e2fa7f3a29a69282148)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:27.658219 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ada0942-e93c-44b9-b831-d5fcc78402b7 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:27.659842 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 563/1121] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 15 19:16:27 2022] GET /identity/v3/users/a9167ddd61e64e2fa7f3a29a69282148 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:16:27.665731 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eba20c6-9975-4b2a-b387-89356ddad16f None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:27.720295 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:27.761807 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eba20c6-9975-4b2a-b387-89356ddad16f None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:27.795081 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec9dfa1-a471-4acf-b6bb-d7bb287cf15d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:27.795875 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec9dfa1-a471-4acf-b6bb-d7bb287cf15d None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:27.796159 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec9dfa1-a471-4acf-b6bb-d7bb287cf15d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:27.796432 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec9dfa1-a471-4acf-b6bb-d7bb287cf15d None admin] PATH_INFO: `/v3/projects/ab5433faa0b14ec29f68a8b8219b16ed` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:27.797605 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec9dfa1-a471-4acf-b6bb-d7bb287cf15d None admin] RBAC: Authorizing `identity:get_project(project_id=ab5433faa0b14ec29f68a8b8219b16ed)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:27.799145 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec9dfa1-a471-4acf-b6bb-d7bb287cf15d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:27.800855 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 559/1122] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 15 19:16:27 2022] GET /identity/v3/projects/ab5433faa0b14ec29f68a8b8219b16ed => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:16:27.806774 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ada0942-e93c-44b9-b831-d5fcc78402b7 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:27.866666 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:27.904200 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ada0942-e93c-44b9-b831-d5fcc78402b7 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:27.936795 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d0d374-fe81-4fde-b04f-b4552c25a0d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:27.937675 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d0d374-fe81-4fde-b04f-b4552c25a0d4 None admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:27.937983 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d0d374-fe81-4fde-b04f-b4552c25a0d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:27.938262 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d0d374-fe81-4fde-b04f-b4552c25a0d4 None admin] PATH_INFO: `/v3/projects/ab5433faa0b14ec29f68a8b8219b16ed/users/a9167ddd61e64e2fa7f3a29a69282148/roles/e97d26621445466d9550261c608db9c5` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:27.940933 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d0d374-fe81-4fde-b04f-b4552c25a0d4 None admin] RBAC: Authorizing `identity:create_grant(project_id=ab5433faa0b14ec29f68a8b8219b16ed, user_id=a9167ddd61e64e2fa7f3a29a69282148, role_id=e97d26621445466d9550261c608db9c5)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:27.942388 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d0d374-fe81-4fde-b04f-b4552c25a0d4 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:28.017801 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 564/1123] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jul 15 19:16:27 2022] PUT /identity/v3/projects/ab5433faa0b14ec29f68a8b8219b16ed/users/a9167ddd61e64e2fa7f3a29a69282148/roles/e97d26621445466d9550261c608db9c5 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:16:29.989790 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b410666-74b4-4153-b685-bad61222067d None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:29.990443 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b410666-74b4-4153-b685-bad61222067d None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:29.990443 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b410666-74b4-4153-b685-bad61222067d None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:29.991713 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 560/1124] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:16:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:16:29.996771 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ddb11e-32b3-432d-9482-0fcfc422ae6a None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:29.997141 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ddb11e-32b3-432d-9482-0fcfc422ae6a None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:29.997332 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ddb11e-32b3-432d-9482-0fcfc422ae6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:30.014137 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-36ddb11e-32b3-432d-9482-0fcfc422ae6a None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:16:30.015316 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-36ddb11e-32b3-432d-9482-0fcfc422ae6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:16:30.079349 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 565/1125] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:16:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:16:30.102619 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b410666-74b4-4153-b685-bad61222067d None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:30.201706 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b410666-74b4-4153-b685-bad61222067d None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:30.224639 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:30.234900 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6968e113-7821-417a-b2f3-85247392bada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:30.235748 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6968e113-7821-417a-b2f3-85247392bada None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:30.249173 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6968e113-7821-417a-b2f3-85247392bada None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:30.249173 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6968e113-7821-417a-b2f3-85247392bada None admin] PATH_INFO: `/v3/roles/e97d26621445466d9550261c608db9c5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:30.249173 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6968e113-7821-417a-b2f3-85247392bada None admin] RBAC: Authorizing `identity:get_role(role_id=e97d26621445466d9550261c608db9c5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:30.249173 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6968e113-7821-417a-b2f3-85247392bada None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:30.249173 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 561/1126] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 15 19:16:30 2022] GET /identity/v3/roles/e97d26621445466d9550261c608db9c5 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:16:30.249630 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ddb11e-32b3-432d-9482-0fcfc422ae6a None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:30.298409 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:30.344823 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ddb11e-32b3-432d-9482-0fcfc422ae6a None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:30.377997 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883aab8d-6daf-4456-b148-a2d1ebc0bcfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:30.378856 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883aab8d-6daf-4456-b148-a2d1ebc0bcfc None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:30.379139 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883aab8d-6daf-4456-b148-a2d1ebc0bcfc None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:30.379414 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883aab8d-6daf-4456-b148-a2d1ebc0bcfc None admin] PATH_INFO: `/v3/users/a9167ddd61e64e2fa7f3a29a69282148` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:30.380595 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883aab8d-6daf-4456-b148-a2d1ebc0bcfc None admin] RBAC: Authorizing `identity:get_user(user_id=a9167ddd61e64e2fa7f3a29a69282148)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:30.382038 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883aab8d-6daf-4456-b148-a2d1ebc0bcfc None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:30.383661 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 566/1127] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 15 19:16:30 2022] GET /identity/v3/users/a9167ddd61e64e2fa7f3a29a69282148 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:16:30.389385 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6968e113-7821-417a-b2f3-85247392bada None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:30.493567 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6968e113-7821-417a-b2f3-85247392bada None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:30.527553 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60bdd5c-b470-42ff-8b76-580ec9936640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:30.528193 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60bdd5c-b470-42ff-8b76-580ec9936640 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:30.528476 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60bdd5c-b470-42ff-8b76-580ec9936640 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:30.528753 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60bdd5c-b470-42ff-8b76-580ec9936640 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:30.584565 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:30.593628 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60bdd5c-b470-42ff-8b76-580ec9936640 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:30.601427 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60bdd5c-b470-42ff-8b76-580ec9936640 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:30.690922 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 562/1128] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 15 19:16:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 19:16:30.696883 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883aab8d-6daf-4456-b148-a2d1ebc0bcfc None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:30.794132 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883aab8d-6daf-4456-b148-a2d1ebc0bcfc None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:30.815957 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:30.829110 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16de49b3-9a92-49ae-a685-661539f594ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:30.830556 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16de49b3-9a92-49ae-a685-661539f594ce None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:30.830834 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16de49b3-9a92-49ae-a685-661539f594ce None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:30.831100 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16de49b3-9a92-49ae-a685-661539f594ce None admin] PATH_INFO: `/v3/projects/ab5433faa0b14ec29f68a8b8219b16ed` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:30.832272 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16de49b3-9a92-49ae-a685-661539f594ce None admin] RBAC: Authorizing `identity:get_project(project_id=ab5433faa0b14ec29f68a8b8219b16ed)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:30.833675 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16de49b3-9a92-49ae-a685-661539f594ce None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:30.835812 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 567/1129] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 15 19:16:30 2022] GET /identity/v3/projects/ab5433faa0b14ec29f68a8b8219b16ed => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:16:30.841531 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60bdd5c-b470-42ff-8b76-580ec9936640 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:30.930714 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:30.939856 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60bdd5c-b470-42ff-8b76-580ec9936640 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:30.974046 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40e2e09-9d26-40da-a407-804435377a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:30.974695 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40e2e09-9d26-40da-a407-804435377a87 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:30.974977 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40e2e09-9d26-40da-a407-804435377a87 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:30.975251 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40e2e09-9d26-40da-a407-804435377a87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:30.976076 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40e2e09-9d26-40da-a407-804435377a87 None admin] RBAC: Adding query filter params (role.id=e97d26621445466d9550261c608db9c5, scope.project.id=ab5433faa0b14ec29f68a8b8219b16ed, user.id=a9167ddd61e64e2fa7f3a29a69282148) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:16:30.976386 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40e2e09-9d26-40da-a407-804435377a87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:30.977847 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40e2e09-9d26-40da-a407-804435377a87 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:30.983558 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 563/1130] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 15 19:16:30 2022] GET /identity/v3/role_assignments?user.id=a9167ddd61e64e2fa7f3a29a69282148&scope.project.id=ab5433faa0b14ec29f68a8b8219b16ed&role.id=e97d26621445466d9550261c608db9c5 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:16:33.011562 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb21d0b7-60e5-486d-bf8e-93c65a956cfa None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:33.012118 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb21d0b7-60e5-486d-bf8e-93c65a956cfa None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:33.012118 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb21d0b7-60e5-486d-bf8e-93c65a956cfa None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:33.013489 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 568/1131] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:16:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:16:33.019668 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ccbee0-225f-43cf-bfac-d336f16d8040 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:33.020095 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ccbee0-225f-43cf-bfac-d336f16d8040 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:33.020195 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ccbee0-225f-43cf-bfac-d336f16d8040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:33.036753 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-a0ccbee0-225f-43cf-bfac-d336f16d8040 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:16:33.037963 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-a0ccbee0-225f-43cf-bfac-d336f16d8040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:16:33.102318 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 564/1132] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:16:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:16:33.125366 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb21d0b7-60e5-486d-bf8e-93c65a956cfa None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:33.224097 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb21d0b7-60e5-486d-bf8e-93c65a956cfa None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:33.259274 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb888ad8-b060-4dce-8db9-fe06655f3baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:33.260053 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb888ad8-b060-4dce-8db9-fe06655f3baf None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:33.260333 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb888ad8-b060-4dce-8db9-fe06655f3baf None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:33.260603 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb888ad8-b060-4dce-8db9-fe06655f3baf None admin] PATH_INFO: `/v3/domains/default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:33.261799 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb888ad8-b060-4dce-8db9-fe06655f3baf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:33.263212 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb888ad8-b060-4dce-8db9-fe06655f3baf None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:33.264828 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 569/1133] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:16:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:16:33.270995 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ccbee0-225f-43cf-bfac-d336f16d8040 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:33.353399 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:33.368017 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ccbee0-225f-43cf-bfac-d336f16d8040 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:33.402289 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee937bd2-423d-4081-84ec-bcde8765a94a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:33.402945 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee937bd2-423d-4081-84ec-bcde8765a94a None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:33.403228 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee937bd2-423d-4081-84ec-bcde8765a94a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:33.403505 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee937bd2-423d-4081-84ec-bcde8765a94a None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:33.404210 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee937bd2-423d-4081-84ec-bcde8765a94a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:33.405636 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee937bd2-423d-4081-84ec-bcde8765a94a None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:33.550831 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 565/1134] 10.222.0.33 () {60 vars in 1141 bytes} [Fri Jul 15 19:16:33 2022] POST /identity/v3/projects => generated 311 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:16:35.573191 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627e4def-ebd9-40d6-86ac-93640ceec34f None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:35.573729 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627e4def-ebd9-40d6-86ac-93640ceec34f None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:35.573729 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627e4def-ebd9-40d6-86ac-93640ceec34f None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:35.575134 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 570/1135] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:16:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:16:35.580339 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db85e889-d87f-4b48-a033-13a116e71da0 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:35.580807 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db85e889-d87f-4b48-a033-13a116e71da0 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:35.580911 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db85e889-d87f-4b48-a033-13a116e71da0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:35.597546 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-db85e889-d87f-4b48-a033-13a116e71da0 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:16:35.598769 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-db85e889-d87f-4b48-a033-13a116e71da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:16:35.662690 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 566/1136] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:16:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:16:35.687535 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627e4def-ebd9-40d6-86ac-93640ceec34f None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:35.743502 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:35.786652 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627e4def-ebd9-40d6-86ac-93640ceec34f None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:35.820065 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a82c04-3809-459e-8d28-53c029d3508b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:35.820849 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a82c04-3809-459e-8d28-53c029d3508b None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:35.821125 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a82c04-3809-459e-8d28-53c029d3508b None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:35.821394 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a82c04-3809-459e-8d28-53c029d3508b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:35.822577 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a82c04-3809-459e-8d28-53c029d3508b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:35.824036 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a82c04-3809-459e-8d28-53c029d3508b None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:35.825583 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 571/1137] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:16:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:16:35.831539 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db85e889-d87f-4b48-a033-13a116e71da0 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:35.895541 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:35.928281 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db85e889-d87f-4b48-a033-13a116e71da0 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:35.960856 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae64579-2cd3-4845-b156-e521ed4db3f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:35.961519 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae64579-2cd3-4845-b156-e521ed4db3f3 None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:35.961830 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae64579-2cd3-4845-b156-e521ed4db3f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:35.962107 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae64579-2cd3-4845-b156-e521ed4db3f3 None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:35.962822 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae64579-2cd3-4845-b156-e521ed4db3f3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:35.964261 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae64579-2cd3-4845-b156-e521ed4db3f3 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:36.074211 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 567/1138] 10.222.0.33 () {60 vars in 1132 bytes} [Fri Jul 15 19:16:35 2022] POST /identity/v3/users => generated 286 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:16:38.103156 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fccf49-10d0-4b99-b77a-22ceefd73eab None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:38.103674 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fccf49-10d0-4b99-b77a-22ceefd73eab None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:38.103674 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fccf49-10d0-4b99-b77a-22ceefd73eab None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:38.105005 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 572/1139] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:16:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:16:38.110033 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db74a872-ba68-4a14-afe5-fefe54a013b2 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:38.110033 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db74a872-ba68-4a14-afe5-fefe54a013b2 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:38.111491 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db74a872-ba68-4a14-afe5-fefe54a013b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:38.128181 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-db74a872-ba68-4a14-afe5-fefe54a013b2 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:16:38.129358 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-db74a872-ba68-4a14-afe5-fefe54a013b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:16:38.196245 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 568/1140] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:16:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:16:38.220129 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fccf49-10d0-4b99-b77a-22ceefd73eab None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:38.325032 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fccf49-10d0-4b99-b77a-22ceefd73eab None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:38.348138 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:38.361135 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c752f7f-a47e-4edf-8c60-2d41f5f34de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:38.361135 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c752f7f-a47e-4edf-8c60-2d41f5f34de8 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:38.361135 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c752f7f-a47e-4edf-8c60-2d41f5f34de8 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:38.361135 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c752f7f-a47e-4edf-8c60-2d41f5f34de8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:38.370243 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c752f7f-a47e-4edf-8c60-2d41f5f34de8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:16:38.378717 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-6c752f7f-a47e-4edf-8c60-2d41f5f34de8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:16:38.378717 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:16:38.378717 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:16:38.378717 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:16:38.378717 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:16:38.378717 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:16:38.378717 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:16:38.378717 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:16:38.378717 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:16:38.378717 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:16:38.378717 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:16:38.378717 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:16:38.378717 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:16:38.378717 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:16:38.378717 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:16:38.378717 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:16:38.379698 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:16:38.379698 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:16:38.379698 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:16:38.379698 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:16:38.379698 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:16:38.379698 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:16:38.379698 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:16:38.379698 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:16:38.379698 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:16:38.379698 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:16:38.379698 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:16:38.379698 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:16:38.379698 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:16:38.379698 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:16:38.379698 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:16:38.379698 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:16:38.380550 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:16:38.380550 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:16:38.380550 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:16:38.380550 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:16:38.380550 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:16:38.380550 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:16:38.380550 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:16:38.380550 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:16:38.380550 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:16:38.380550 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:16:38.380550 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:16:38.380550 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:16:38.380550 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 573/1141] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jul 15 19:16:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:16:38.386099 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db74a872-ba68-4a14-afe5-fefe54a013b2 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:38.439292 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:38.481059 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db74a872-ba68-4a14-afe5-fefe54a013b2 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:38.514837 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4affee39-07b7-4017-8cc9-ba75bc4ce4f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:38.515490 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4affee39-07b7-4017-8cc9-ba75bc4ce4f9 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:38.515766 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4affee39-07b7-4017-8cc9-ba75bc4ce4f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:38.516041 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4affee39-07b7-4017-8cc9-ba75bc4ce4f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:38.517305 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4affee39-07b7-4017-8cc9-ba75bc4ce4f9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:16:38.517629 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4affee39-07b7-4017-8cc9-ba75bc4ce4f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:38.519329 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4affee39-07b7-4017-8cc9-ba75bc4ce4f9 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:38.529740 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 569/1142] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 15 19:16:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:16:38.539224 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c752f7f-a47e-4edf-8c60-2d41f5f34de8 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:38.642253 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c752f7f-a47e-4edf-8c60-2d41f5f34de8 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:38.675839 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10114c9-83b9-4569-8c6b-525c2d43d205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:38.676697 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10114c9-83b9-4569-8c6b-525c2d43d205 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:38.676974 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10114c9-83b9-4569-8c6b-525c2d43d205 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:38.677251 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10114c9-83b9-4569-8c6b-525c2d43d205 None admin] PATH_INFO: `/v3/users/11d2dff33bc14c92b3159cb0e9b42886` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:38.690338 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10114c9-83b9-4569-8c6b-525c2d43d205 None admin] RBAC: Authorizing `identity:get_user(user_id=11d2dff33bc14c92b3159cb0e9b42886)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:38.691766 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10114c9-83b9-4569-8c6b-525c2d43d205 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:38.693422 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 574/1143] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 15 19:16:38 2022] GET /identity/v3/users/11d2dff33bc14c92b3159cb0e9b42886 => generated 286 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:16:38.699225 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4affee39-07b7-4017-8cc9-ba75bc4ce4f9 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:38.797416 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4affee39-07b7-4017-8cc9-ba75bc4ce4f9 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:38.814645 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:38.832397 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569a1361-34e0-4a37-a8bc-9afed7b969ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:38.833033 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569a1361-34e0-4a37-a8bc-9afed7b969ac None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:38.833319 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569a1361-34e0-4a37-a8bc-9afed7b969ac None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:38.833599 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569a1361-34e0-4a37-a8bc-9afed7b969ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:38.900790 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569a1361-34e0-4a37-a8bc-9afed7b969ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:38.902187 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569a1361-34e0-4a37-a8bc-9afed7b969ac None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:38.998344 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 570/1144] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 15 19:16:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 19:16:39.004234 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10114c9-83b9-4569-8c6b-525c2d43d205 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:39.046567 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:39.101295 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10114c9-83b9-4569-8c6b-525c2d43d205 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:39.133345 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c64ecec-58ad-4700-b170-4cba0d51e26f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:39.134223 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c64ecec-58ad-4700-b170-4cba0d51e26f None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:39.134498 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c64ecec-58ad-4700-b170-4cba0d51e26f None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:39.134763 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c64ecec-58ad-4700-b170-4cba0d51e26f None admin] PATH_INFO: `/v3/projects/d253abcd61a8428ab05dd355d2921ece` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:39.135914 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c64ecec-58ad-4700-b170-4cba0d51e26f None admin] RBAC: Authorizing `identity:get_project(project_id=d253abcd61a8428ab05dd355d2921ece)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:39.137650 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c64ecec-58ad-4700-b170-4cba0d51e26f None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:39.139341 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 575/1145] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 15 19:16:39 2022] GET /identity/v3/projects/d253abcd61a8428ab05dd355d2921ece => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:16:39.146206 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569a1361-34e0-4a37-a8bc-9afed7b969ac None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:39.163728 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:39.242966 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569a1361-34e0-4a37-a8bc-9afed7b969ac None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:39.276013 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31abfd2-1e8e-499c-97cc-245ff05950e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:39.276657 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31abfd2-1e8e-499c-97cc-245ff05950e7 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:39.276940 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31abfd2-1e8e-499c-97cc-245ff05950e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:39.277216 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31abfd2-1e8e-499c-97cc-245ff05950e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:39.278075 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31abfd2-1e8e-499c-97cc-245ff05950e7 None admin] RBAC: Adding query filter params (role.id=4318f86ba9f74cd4a0bef1c311276657, scope.project.id=d253abcd61a8428ab05dd355d2921ece, user.id=11d2dff33bc14c92b3159cb0e9b42886) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:16:39.278388 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31abfd2-1e8e-499c-97cc-245ff05950e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:39.279812 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31abfd2-1e8e-499c-97cc-245ff05950e7 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:39.285249 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 571/1146] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 15 19:16:39 2022] GET /identity/v3/role_assignments?user.id=11d2dff33bc14c92b3159cb0e9b42886&scope.project.id=d253abcd61a8428ab05dd355d2921ece&role.id=4318f86ba9f74cd4a0bef1c311276657 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:16:41.255917 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ea3ea1-87b4-4fc2-bca6-0f7ce71ebdf5 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:41.256457 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ea3ea1-87b4-4fc2-bca6-0f7ce71ebdf5 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:41.256457 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ea3ea1-87b4-4fc2-bca6-0f7ce71ebdf5 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:41.257807 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 576/1147] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:16:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:16:41.263454 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d75556-dd91-4a64-8b3d-0b33993a8682 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:41.263913 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d75556-dd91-4a64-8b3d-0b33993a8682 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:41.264008 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d75556-dd91-4a64-8b3d-0b33993a8682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:41.282947 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-07d75556-dd91-4a64-8b3d-0b33993a8682 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:16:41.284160 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-07d75556-dd91-4a64-8b3d-0b33993a8682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:16:41.352566 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 572/1148] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:16:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:16:41.374773 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ea3ea1-87b4-4fc2-bca6-0f7ce71ebdf5 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:41.474250 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ea3ea1-87b4-4fc2-bca6-0f7ce71ebdf5 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:41.484370 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:41.507245 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b8d9ca-65de-49fa-b30a-3bffd5ef2e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:41.507961 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b8d9ca-65de-49fa-b30a-3bffd5ef2e2b None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:41.508239 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b8d9ca-65de-49fa-b30a-3bffd5ef2e2b None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:41.508510 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b8d9ca-65de-49fa-b30a-3bffd5ef2e2b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:41.518835 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b8d9ca-65de-49fa-b30a-3bffd5ef2e2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:16:41.526886 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-63b8d9ca-65de-49fa-b30a-3bffd5ef2e2b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:16:41.526886 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:16:41.526886 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:16:41.526886 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:16:41.526886 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:16:41.526886 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:16:41.526886 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:16:41.526886 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:16:41.526886 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:16:41.526886 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:16:41.526886 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:16:41.526886 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:16:41.526886 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:16:41.526886 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:16:41.526886 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:16:41.526886 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:16:41.527992 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:16:41.527992 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:16:41.527992 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:16:41.527992 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:16:41.527992 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:16:41.527992 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:16:41.527992 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:16:41.527992 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:16:41.527992 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:16:41.527992 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:16:41.527992 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:16:41.527992 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:16:41.527992 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:16:41.527992 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:16:41.527992 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:16:41.527992 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:16:41.529090 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:16:41.529090 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:16:41.529090 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:16:41.529090 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:16:41.529090 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:16:41.529090 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:16:41.529090 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:16:41.529090 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:16:41.529090 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:16:41.529090 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:16:41.529090 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:16:41.529090 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:16:41.529090 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 577/1149] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jul 15 19:16:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:16:41.535148 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d75556-dd91-4a64-8b3d-0b33993a8682 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:41.546598 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:41.639475 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d75556-dd91-4a64-8b3d-0b33993a8682 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:41.673247 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db050b19-3468-44a8-af9c-b3b587ff11eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:41.673935 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db050b19-3468-44a8-af9c-b3b587ff11eb None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:41.674220 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db050b19-3468-44a8-af9c-b3b587ff11eb None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:41.674494 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db050b19-3468-44a8-af9c-b3b587ff11eb None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:41.675232 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db050b19-3468-44a8-af9c-b3b587ff11eb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:16:41.675535 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db050b19-3468-44a8-af9c-b3b587ff11eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:41.676909 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db050b19-3468-44a8-af9c-b3b587ff11eb None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:41.684742 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 573/1150] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 15 19:16:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:16:41.690668 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b8d9ca-65de-49fa-b30a-3bffd5ef2e2b None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:41.788660 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b8d9ca-65de-49fa-b30a-3bffd5ef2e2b None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:41.821255 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b348156-37c9-4eef-9a45-d2c583f59666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:41.822147 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b348156-37c9-4eef-9a45-d2c583f59666 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:41.822432 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b348156-37c9-4eef-9a45-d2c583f59666 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:41.822711 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b348156-37c9-4eef-9a45-d2c583f59666 None admin] PATH_INFO: `/v3/users/11d2dff33bc14c92b3159cb0e9b42886` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:41.823898 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b348156-37c9-4eef-9a45-d2c583f59666 None admin] RBAC: Authorizing `identity:get_user(user_id=11d2dff33bc14c92b3159cb0e9b42886)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:41.825318 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b348156-37c9-4eef-9a45-d2c583f59666 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:41.826984 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 578/1151] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 15 19:16:41 2022] GET /identity/v3/users/11d2dff33bc14c92b3159cb0e9b42886 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:16:41.833279 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db050b19-3468-44a8-af9c-b3b587ff11eb None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:41.903204 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:41.931500 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db050b19-3468-44a8-af9c-b3b587ff11eb None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:41.965091 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6828d07e-8a9b-4281-a83e-1eeffac9200d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:41.965974 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6828d07e-8a9b-4281-a83e-1eeffac9200d None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:41.966258 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6828d07e-8a9b-4281-a83e-1eeffac9200d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:41.966532 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6828d07e-8a9b-4281-a83e-1eeffac9200d None admin] PATH_INFO: `/v3/projects/d253abcd61a8428ab05dd355d2921ece` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:41.967714 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6828d07e-8a9b-4281-a83e-1eeffac9200d None admin] RBAC: Authorizing `identity:get_project(project_id=d253abcd61a8428ab05dd355d2921ece)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:41.969151 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6828d07e-8a9b-4281-a83e-1eeffac9200d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:41.970912 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 574/1152] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 15 19:16:41 2022] GET /identity/v3/projects/d253abcd61a8428ab05dd355d2921ece => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:16:41.976816 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b348156-37c9-4eef-9a45-d2c583f59666 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:42.015733 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:42.073659 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b348156-37c9-4eef-9a45-d2c583f59666 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:42.106186 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8fc216-7ffc-4db5-ba8f-1fcbc8e8366a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:42.107058 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8fc216-7ffc-4db5-ba8f-1fcbc8e8366a None admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:42.107338 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8fc216-7ffc-4db5-ba8f-1fcbc8e8366a None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:42.107610 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8fc216-7ffc-4db5-ba8f-1fcbc8e8366a None admin] PATH_INFO: `/v3/projects/d253abcd61a8428ab05dd355d2921ece/users/11d2dff33bc14c92b3159cb0e9b42886/roles/4318f86ba9f74cd4a0bef1c311276657` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:42.109539 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8fc216-7ffc-4db5-ba8f-1fcbc8e8366a None admin] RBAC: Authorizing `identity:create_grant(project_id=d253abcd61a8428ab05dd355d2921ece, user_id=11d2dff33bc14c92b3159cb0e9b42886, role_id=4318f86ba9f74cd4a0bef1c311276657)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:42.111509 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8fc216-7ffc-4db5-ba8f-1fcbc8e8366a None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:42.206353 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 579/1153] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jul 15 19:16:41 2022] PUT /identity/v3/projects/d253abcd61a8428ab05dd355d2921ece/users/11d2dff33bc14c92b3159cb0e9b42886/roles/4318f86ba9f74cd4a0bef1c311276657 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:16:44.182772 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1413ee-a827-4078-a1cb-7145c989832a None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:44.183297 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1413ee-a827-4078-a1cb-7145c989832a None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:44.183297 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1413ee-a827-4078-a1cb-7145c989832a None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:44.184597 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 575/1154] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:16:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:16:44.189942 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3756a0f9-dbd6-4676-9625-82dbdfb66bae None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:44.190363 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3756a0f9-dbd6-4676-9625-82dbdfb66bae None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:44.190609 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3756a0f9-dbd6-4676-9625-82dbdfb66bae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:44.207159 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-3756a0f9-dbd6-4676-9625-82dbdfb66bae None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:16:44.208337 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-3756a0f9-dbd6-4676-9625-82dbdfb66bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:16:44.273091 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 580/1155] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:16:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:16:44.296711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1413ee-a827-4078-a1cb-7145c989832a None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:44.396897 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1413ee-a827-4078-a1cb-7145c989832a None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:44.429324 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3892522e-bd0b-49af-8104-698dc920b4c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:44.430132 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3892522e-bd0b-49af-8104-698dc920b4c1 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:44.430412 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3892522e-bd0b-49af-8104-698dc920b4c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:44.430682 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3892522e-bd0b-49af-8104-698dc920b4c1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:44.438152 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:44.442046 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3892522e-bd0b-49af-8104-698dc920b4c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:16:44.449445 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-3892522e-bd0b-49af-8104-698dc920b4c1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:16:44.449445 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:16:44.449445 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:16:44.449445 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:16:44.449445 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:16:44.449445 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:16:44.449445 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:16:44.449445 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:16:44.449445 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:16:44.449445 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:16:44.449445 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:16:44.449445 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:16:44.449445 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:16:44.449445 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:16:44.449445 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:16:44.449445 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:16:44.450457 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:16:44.450457 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:16:44.450457 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:16:44.450457 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:16:44.450457 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:16:44.450457 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:16:44.450457 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:16:44.450457 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:16:44.450457 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:16:44.450457 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:16:44.450457 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:16:44.450457 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:16:44.450457 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:16:44.450457 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:16:44.450457 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:16:44.450457 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:16:44.451360 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:16:44.451360 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:16:44.451360 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:16:44.451360 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:16:44.451360 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:16:44.451360 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:16:44.451360 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:16:44.451360 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:16:44.451360 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:16:44.451360 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:16:44.451360 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:16:44.451360 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:16:44.451360 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 576/1156] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jul 15 19:16:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:16:44.457201 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3756a0f9-dbd6-4676-9625-82dbdfb66bae None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:44.483936 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:44.554108 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3756a0f9-dbd6-4676-9625-82dbdfb66bae None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:44.591856 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aade0f-e28a-46bb-ba13-f9bed3373d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:44.592521 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aade0f-e28a-46bb-ba13-f9bed3373d7b None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:44.592805 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aade0f-e28a-46bb-ba13-f9bed3373d7b None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:44.593242 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aade0f-e28a-46bb-ba13-f9bed3373d7b None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:44.594449 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aade0f-e28a-46bb-ba13-f9bed3373d7b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:16:44.594710 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aade0f-e28a-46bb-ba13-f9bed3373d7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:44.596136 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aade0f-e28a-46bb-ba13-f9bed3373d7b None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:44.604583 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 581/1157] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 15 19:16:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:16:44.610965 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3892522e-bd0b-49af-8104-698dc920b4c1 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:44.706287 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3892522e-bd0b-49af-8104-698dc920b4c1 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:44.739150 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d056d7-b88b-484c-a9b3-0c0e8becf4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:44.740021 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d056d7-b88b-484c-a9b3-0c0e8becf4be None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:44.740321 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d056d7-b88b-484c-a9b3-0c0e8becf4be None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:44.740591 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d056d7-b88b-484c-a9b3-0c0e8becf4be None admin] PATH_INFO: `/v3/users/11d2dff33bc14c92b3159cb0e9b42886` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:44.741786 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d056d7-b88b-484c-a9b3-0c0e8becf4be None admin] RBAC: Authorizing `identity:get_user(user_id=11d2dff33bc14c92b3159cb0e9b42886)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:44.743217 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d056d7-b88b-484c-a9b3-0c0e8becf4be None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:44.744880 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 577/1158] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 15 19:16:44 2022] GET /identity/v3/users/11d2dff33bc14c92b3159cb0e9b42886 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:16:44.750636 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aade0f-e28a-46bb-ba13-f9bed3373d7b None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:44.837996 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:44.847590 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aade0f-e28a-46bb-ba13-f9bed3373d7b None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:44.880336 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627548b4-3297-48bf-80bd-23c5aaa3fb1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:44.880973 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627548b4-3297-48bf-80bd-23c5aaa3fb1c None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:44.881255 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627548b4-3297-48bf-80bd-23c5aaa3fb1c None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:44.881529 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627548b4-3297-48bf-80bd-23c5aaa3fb1c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:44.945676 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627548b4-3297-48bf-80bd-23c5aaa3fb1c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:44.947124 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627548b4-3297-48bf-80bd-23c5aaa3fb1c None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:45.036596 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:45.042079 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 582/1159] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 15 19:16:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 19:16:45.048557 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d056d7-b88b-484c-a9b3-0c0e8becf4be None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:45.113888 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:45.146204 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d056d7-b88b-484c-a9b3-0c0e8becf4be None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:45.178982 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961be9fe-5af7-4439-9c3a-39fdc446d32d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:45.179835 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961be9fe-5af7-4439-9c3a-39fdc446d32d None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:45.180118 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961be9fe-5af7-4439-9c3a-39fdc446d32d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:45.180390 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961be9fe-5af7-4439-9c3a-39fdc446d32d None admin] PATH_INFO: `/v3/projects/d253abcd61a8428ab05dd355d2921ece` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:45.181566 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961be9fe-5af7-4439-9c3a-39fdc446d32d None admin] RBAC: Authorizing `identity:get_project(project_id=d253abcd61a8428ab05dd355d2921ece)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:45.183041 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961be9fe-5af7-4439-9c3a-39fdc446d32d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:45.184740 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 578/1160] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 15 19:16:45 2022] GET /identity/v3/projects/d253abcd61a8428ab05dd355d2921ece => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:16:45.191515 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627548b4-3297-48bf-80bd-23c5aaa3fb1c None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:45.288382 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627548b4-3297-48bf-80bd-23c5aaa3fb1c None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:45.321561 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dca7e58-6f51-478e-9e8c-4c93d909ccf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:45.322231 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dca7e58-6f51-478e-9e8c-4c93d909ccf6 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:45.322520 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dca7e58-6f51-478e-9e8c-4c93d909ccf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:45.322797 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dca7e58-6f51-478e-9e8c-4c93d909ccf6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:45.323626 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dca7e58-6f51-478e-9e8c-4c93d909ccf6 None admin] RBAC: Adding query filter params (role.id=4318f86ba9f74cd4a0bef1c311276657, scope.project.id=d253abcd61a8428ab05dd355d2921ece, user.id=11d2dff33bc14c92b3159cb0e9b42886) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:16:45.323940 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dca7e58-6f51-478e-9e8c-4c93d909ccf6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:45.325352 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dca7e58-6f51-478e-9e8c-4c93d909ccf6 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:45.331604 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 583/1161] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 15 19:16:45 2022] GET /identity/v3/role_assignments?user.id=11d2dff33bc14c92b3159cb0e9b42886&scope.project.id=d253abcd61a8428ab05dd355d2921ece&role.id=4318f86ba9f74cd4a0bef1c311276657 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:16:47.358388 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4040213-e65a-4c8f-9824-e08200d1f558 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:47.358910 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4040213-e65a-4c8f-9824-e08200d1f558 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:47.359027 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4040213-e65a-4c8f-9824-e08200d1f558 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:47.360854 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 579/1162] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:16:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:16:47.366042 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0114afcc-c8a8-42dc-b6dc-9f0b00e3a3b7 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:47.366481 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0114afcc-c8a8-42dc-b6dc-9f0b00e3a3b7 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:47.366599 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0114afcc-c8a8-42dc-b6dc-9f0b00e3a3b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:47.383525 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-0114afcc-c8a8-42dc-b6dc-9f0b00e3a3b7 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:16:47.384743 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-0114afcc-c8a8-42dc-b6dc-9f0b00e3a3b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:16:47.432286 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:47.449519 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 584/1163] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:16:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 19:16:47.471201 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4040213-e65a-4c8f-9824-e08200d1f558 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:47.571025 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4040213-e65a-4c8f-9824-e08200d1f558 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:47.604985 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ca7ac8-10b9-40aa-8c5e-e75c58306900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:47.605626 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ca7ac8-10b9-40aa-8c5e-e75c58306900 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:47.605941 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ca7ac8-10b9-40aa-8c5e-e75c58306900 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:47.606221 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ca7ac8-10b9-40aa-8c5e-e75c58306900 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:47.609725 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:47.676025 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ca7ac8-10b9-40aa-8c5e-e75c58306900 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:47.677495 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ca7ac8-10b9-40aa-8c5e-e75c58306900 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:47.773284 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 580/1164] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 15 19:16:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 19:16:47.779220 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0114afcc-c8a8-42dc-b6dc-9f0b00e3a3b7 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:47.875220 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0114afcc-c8a8-42dc-b6dc-9f0b00e3a3b7 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:47.908904 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800d0118-3103-4aab-ba29-a2d16a5e74f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:47.909686 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800d0118-3103-4aab-ba29-a2d16a5e74f2 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:47.909990 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800d0118-3103-4aab-ba29-a2d16a5e74f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:47.910259 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800d0118-3103-4aab-ba29-a2d16a5e74f2 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:47.916648 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800d0118-3103-4aab-ba29-a2d16a5e74f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:47.918096 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800d0118-3103-4aab-ba29-a2d16a5e74f2 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:47.931834 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-800d0118-3103-4aab-ba29-a2d16a5e74f2 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 15 19:16:47.931834 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:16:47.931834 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 15 19:16:47.931834 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 15 19:16:47.931834 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:16:47.931834 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:16:47.931834 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:16:47.931834 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:16:47.931834 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 15 19:16:47.931834 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:16:47.931834 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 19:16:47.931834 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:16:47.931834 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:16:47.931834 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:16:47.931834 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:16:47.931834 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:16:47.931834 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:16:47.933040 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:16:47.933040 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:16:47.933040 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:16:47.933040 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:16:47.933040 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:16:47.933040 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:16:47.933040 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:16:47.933040 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:16:47.933040 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:16:47.933040 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:16:47.933040 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:16:47.933040 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:16:47.933040 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:16:47.933040 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:16:47.933040 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:16:47.933040 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:16:47.933919 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:16:47.933919 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:16:47.933919 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:16:47.933919 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:16:47.933919 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:16:47.933919 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:16:47.933919 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:16:47.933919 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:16:47.933919 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:16:47.933919 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:16:47.933919 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 15 19:16:47.933919 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:16:47.933919 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 15 19:16:47.933919 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:16:47.933919 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 585/1165] 10.222.0.33 () {56 vars in 1117 bytes} [Fri Jul 15 19:16:47 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:16:47.939583 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ca7ac8-10b9-40aa-8c5e-e75c58306900 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:47.977365 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:48.042740 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ca7ac8-10b9-40aa-8c5e-e75c58306900 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:48.075609 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54da01a5-ba4c-4f0a-bd1f-202002687a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:48.076266 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da01a5-ba4c-4f0a-bd1f-202002687a49 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:48.076547 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da01a5-ba4c-4f0a-bd1f-202002687a49 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:48.076822 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da01a5-ba4c-4f0a-bd1f-202002687a49 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:48.077534 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54da01a5-ba4c-4f0a-bd1f-202002687a49 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:16:48.077865 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54da01a5-ba4c-4f0a-bd1f-202002687a49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:48.079831 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54da01a5-ba4c-4f0a-bd1f-202002687a49 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:48.086821 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 581/1166] 10.222.0.33 () {56 vars in 1115 bytes} [Fri Jul 15 19:16:47 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:16:50.062649 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf8c99d-dc1a-42fe-9db4-d79b50947d61 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:50.063687 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf8c99d-dc1a-42fe-9db4-d79b50947d61 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:50.063687 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf8c99d-dc1a-42fe-9db4-d79b50947d61 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:50.065011 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 586/1167] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:16:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:16:50.070305 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938359ae-cb51-4640-9331-263f386743df None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:50.070725 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938359ae-cb51-4640-9331-263f386743df None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:50.070872 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938359ae-cb51-4640-9331-263f386743df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:50.087853 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-938359ae-cb51-4640-9331-263f386743df None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:16:50.089043 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-938359ae-cb51-4640-9331-263f386743df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:16:50.153637 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 582/1168] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:16:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:16:50.175903 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf8c99d-dc1a-42fe-9db4-d79b50947d61 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:50.222230 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:50.275845 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf8c99d-dc1a-42fe-9db4-d79b50947d61 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:50.308399 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a484d3c-ac40-40ec-b31c-5b2e1670e115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:50.309040 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a484d3c-ac40-40ec-b31c-5b2e1670e115 None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:50.309320 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a484d3c-ac40-40ec-b31c-5b2e1670e115 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:50.309593 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a484d3c-ac40-40ec-b31c-5b2e1670e115 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:50.310353 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a484d3c-ac40-40ec-b31c-5b2e1670e115 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:50.311750 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a484d3c-ac40-40ec-b31c-5b2e1670e115 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:50.410607 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 587/1169] 10.222.0.33 () {60 vars in 1138 bytes} [Fri Jul 15 19:16:50 2022] POST /identity/v3/domains => generated 267 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:16:52.444451 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52883c3a-fa40-46d6-8680-1e6df4fe2aa6 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:52.445018 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52883c3a-fa40-46d6-8680-1e6df4fe2aa6 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:52.445018 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52883c3a-fa40-46d6-8680-1e6df4fe2aa6 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:52.446418 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 583/1170] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:16:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:16:52.451589 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a35e2c1-567e-4f99-80cc-e47745d00a15 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:52.451983 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a35e2c1-567e-4f99-80cc-e47745d00a15 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:52.452139 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a35e2c1-567e-4f99-80cc-e47745d00a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:52.469020 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-3a35e2c1-567e-4f99-80cc-e47745d00a15 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:16:52.470283 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-3a35e2c1-567e-4f99-80cc-e47745d00a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:16:52.535023 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 588/1171] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:16:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:16:52.557654 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52883c3a-fa40-46d6-8680-1e6df4fe2aa6 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:52.588463 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:52.660286 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52883c3a-fa40-46d6-8680-1e6df4fe2aa6 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:52.693330 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db572c2-3a47-4ad8-89da-e35dabf56bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:52.694225 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db572c2-3a47-4ad8-89da-e35dabf56bca None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:52.694554 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db572c2-3a47-4ad8-89da-e35dabf56bca None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:52.694776 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db572c2-3a47-4ad8-89da-e35dabf56bca None admin] PATH_INFO: `/v3/domains/b3dc9aefaf5946fcb1808a6fcb5cb9c9` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:52.705341 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db572c2-3a47-4ad8-89da-e35dabf56bca None admin] RBAC: Authorizing `identity:get_domain(domain_id=b3dc9aefaf5946fcb1808a6fcb5cb9c9)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:52.706800 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db572c2-3a47-4ad8-89da-e35dabf56bca None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:52.708437 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 584/1172] 10.222.0.33 () {56 vars in 1183 bytes} [Fri Jul 15 19:16:52 2022] GET /identity/v3/domains/b3dc9aefaf5946fcb1808a6fcb5cb9c9 => generated 267 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:16:52.714603 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a35e2c1-567e-4f99-80cc-e47745d00a15 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:52.747317 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:52.811704 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a35e2c1-567e-4f99-80cc-e47745d00a15 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:52.845822 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b215b8b-5d8e-4e59-a5fa-ff72363ee9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:52.846471 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b215b8b-5d8e-4e59-a5fa-ff72363ee9a9 None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:52.846756 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b215b8b-5d8e-4e59-a5fa-ff72363ee9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:52.847034 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b215b8b-5d8e-4e59-a5fa-ff72363ee9a9 None admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:52.847744 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b215b8b-5d8e-4e59-a5fa-ff72363ee9a9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:52.849162 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b215b8b-5d8e-4e59-a5fa-ff72363ee9a9 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:52.927386 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 589/1173] 10.222.0.33 () {60 vars in 1141 bytes} [Fri Jul 15 19:16:52 2022] POST /identity/v3/projects => generated 361 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:16:54.949801 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef8225-4489-4990-87cb-4181aa3977ee None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:54.950338 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef8225-4489-4990-87cb-4181aa3977ee None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:54.950338 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef8225-4489-4990-87cb-4181aa3977ee None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:54.951730 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 585/1174] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:16:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:16:54.957467 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89103d00-d130-463b-8f23-843e20aa8971 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:54.957874 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89103d00-d130-463b-8f23-843e20aa8971 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:54.958073 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89103d00-d130-463b-8f23-843e20aa8971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:54.975406 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-89103d00-d130-463b-8f23-843e20aa8971 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:16:54.976593 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-89103d00-d130-463b-8f23-843e20aa8971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:16:55.040664 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 590/1175] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:16:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:16:55.065579 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deef8225-4489-4990-87cb-4181aa3977ee None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:55.154491 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:55.164430 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deef8225-4489-4990-87cb-4181aa3977ee None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:55.197990 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391a198d-ec03-492b-b9b4-ca99b87fea17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:55.198840 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391a198d-ec03-492b-b9b4-ca99b87fea17 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:55.199121 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391a198d-ec03-492b-b9b4-ca99b87fea17 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:55.199401 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391a198d-ec03-492b-b9b4-ca99b87fea17 None admin] PATH_INFO: `/v3/domains/b3dc9aefaf5946fcb1808a6fcb5cb9c9` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:55.200579 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391a198d-ec03-492b-b9b4-ca99b87fea17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=b3dc9aefaf5946fcb1808a6fcb5cb9c9)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:55.202048 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391a198d-ec03-492b-b9b4-ca99b87fea17 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:55.203719 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 586/1176] 10.222.0.33 () {56 vars in 1183 bytes} [Fri Jul 15 19:16:55 2022] GET /identity/v3/domains/b3dc9aefaf5946fcb1808a6fcb5cb9c9 => generated 267 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:16:55.209493 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89103d00-d130-463b-8f23-843e20aa8971 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:55.226552 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:55.306697 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89103d00-d130-463b-8f23-843e20aa8971 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:55.340439 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59a7c75-1456-477f-8db2-d5a6067815a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:55.341040 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a7c75-1456-477f-8db2-d5a6067815a6 None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:55.341322 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a7c75-1456-477f-8db2-d5a6067815a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:55.341593 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a7c75-1456-477f-8db2-d5a6067815a6 None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:55.342344 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59a7c75-1456-477f-8db2-d5a6067815a6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:55.343759 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59a7c75-1456-477f-8db2-d5a6067815a6 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:55.440311 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 591/1177] 10.222.0.33 () {60 vars in 1132 bytes} [Fri Jul 15 19:16:55 2022] POST /identity/v3/users => generated 311 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:16:57.489294 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c582fd45-e377-47ed-ad99-96a3da42f922 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:57.489873 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c582fd45-e377-47ed-ad99-96a3da42f922 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:57.489873 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c582fd45-e377-47ed-ad99-96a3da42f922 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:57.491219 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 587/1178] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:16:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:16:57.496571 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2154f19b-ff9e-4f2b-b2e1-5fd44085f80c None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:57.496987 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2154f19b-ff9e-4f2b-b2e1-5fd44085f80c None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:57.497136 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2154f19b-ff9e-4f2b-b2e1-5fd44085f80c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:57.514013 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-2154f19b-ff9e-4f2b-b2e1-5fd44085f80c None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:16:57.515205 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-2154f19b-ff9e-4f2b-b2e1-5fd44085f80c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:16:57.580168 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 592/1179] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:16:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:16:57.604226 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c582fd45-e377-47ed-ad99-96a3da42f922 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:57.703417 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c582fd45-e377-47ed-ad99-96a3da42f922 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:57.740257 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2512984c-48f6-4010-accf-120e7fe93bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:57.740968 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2512984c-48f6-4010-accf-120e7fe93bc9 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:57.741250 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2512984c-48f6-4010-accf-120e7fe93bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:57.741522 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2512984c-48f6-4010-accf-120e7fe93bc9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:57.751412 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2512984c-48f6-4010-accf-120e7fe93bc9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:16:57.759045 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-2512984c-48f6-4010-accf-120e7fe93bc9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:16:57.759045 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:16:57.759045 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:16:57.759045 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:16:57.759045 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:16:57.759045 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:16:57.759045 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:16:57.759045 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:16:57.759045 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:16:57.759045 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:16:57.759045 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:16:57.759045 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:16:57.759045 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:16:57.759045 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:16:57.759045 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:16:57.759045 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:16:57.760195 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:16:57.760195 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:16:57.760195 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:16:57.760195 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:16:57.760195 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:16:57.760195 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:16:57.760195 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:16:57.760195 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:16:57.760195 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:16:57.760195 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:16:57.760195 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:16:57.760195 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:16:57.760195 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:16:57.760195 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:16:57.760195 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:16:57.760195 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:16:57.761451 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:16:57.761451 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:16:57.761451 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:16:57.761451 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:16:57.761451 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:16:57.761451 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:16:57.761451 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:16:57.761451 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:16:57.761451 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:16:57.761451 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:16:57.761451 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:16:57.761451 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:16:57.761451 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 588/1180] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jul 15 19:16:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:16:57.766678 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2154f19b-ff9e-4f2b-b2e1-5fd44085f80c None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:57.769085 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:57.864020 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2154f19b-ff9e-4f2b-b2e1-5fd44085f80c None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:57.897149 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40e63d7-cfd7-4b0e-9263-bd7dfe0f1dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:57.897830 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40e63d7-cfd7-4b0e-9263-bd7dfe0f1dd9 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:57.898114 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40e63d7-cfd7-4b0e-9263-bd7dfe0f1dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:57.898388 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40e63d7-cfd7-4b0e-9263-bd7dfe0f1dd9 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:57.899121 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40e63d7-cfd7-4b0e-9263-bd7dfe0f1dd9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:16:57.899455 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40e63d7-cfd7-4b0e-9263-bd7dfe0f1dd9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:57.900902 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40e63d7-cfd7-4b0e-9263-bd7dfe0f1dd9 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:57.908741 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 593/1181] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 15 19:16:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:16:57.914561 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2512984c-48f6-4010-accf-120e7fe93bc9 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:57.934105 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:58.012945 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2512984c-48f6-4010-accf-120e7fe93bc9 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:58.044787 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f16d06b-1f99-4a3e-a420-19affdadd4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:58.045637 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f16d06b-1f99-4a3e-a420-19affdadd4a2 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:58.045941 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f16d06b-1f99-4a3e-a420-19affdadd4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:58.046214 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f16d06b-1f99-4a3e-a420-19affdadd4a2 None admin] PATH_INFO: `/v3/users/ca547291e8574b33856f0b453b080157` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:58.059277 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f16d06b-1f99-4a3e-a420-19affdadd4a2 None admin] RBAC: Authorizing `identity:get_user(user_id=ca547291e8574b33856f0b453b080157)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:58.060696 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f16d06b-1f99-4a3e-a420-19affdadd4a2 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:58.062356 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 589/1182] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 15 19:16:57 2022] GET /identity/v3/users/ca547291e8574b33856f0b453b080157 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:16:58.068343 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40e63d7-cfd7-4b0e-9263-bd7dfe0f1dd9 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:58.128418 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:58.164835 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40e63d7-cfd7-4b0e-9263-bd7dfe0f1dd9 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:58.198453 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7193629b-f05b-48be-90bb-619c8b5b19dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:58.199080 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7193629b-f05b-48be-90bb-619c8b5b19dc None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:58.199357 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7193629b-f05b-48be-90bb-619c8b5b19dc None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:58.199629 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7193629b-f05b-48be-90bb-619c8b5b19dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:58.263573 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7193629b-f05b-48be-90bb-619c8b5b19dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:58.264927 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7193629b-f05b-48be-90bb-619c8b5b19dc None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:58.323278 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:58.360131 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 594/1183] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 15 19:16:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 19:16:58.365885 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f16d06b-1f99-4a3e-a420-19affdadd4a2 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:58.440090 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:58.461543 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f16d06b-1f99-4a3e-a420-19affdadd4a2 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:58.525928 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d67113-bfad-4707-a103-d2c6c4d5692c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:58.526714 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d67113-bfad-4707-a103-d2c6c4d5692c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:58.526990 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d67113-bfad-4707-a103-d2c6c4d5692c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:58.527261 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d67113-bfad-4707-a103-d2c6c4d5692c None admin] PATH_INFO: `/v3/projects/1dfa624cf8b64a7b858645cab57a1589` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:58.528421 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d67113-bfad-4707-a103-d2c6c4d5692c None admin] RBAC: Authorizing `identity:get_project(project_id=1dfa624cf8b64a7b858645cab57a1589)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:58.529858 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d67113-bfad-4707-a103-d2c6c4d5692c None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:58.532149 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 590/1184] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 15 19:16:58 2022] GET /identity/v3/projects/1dfa624cf8b64a7b858645cab57a1589 => generated 361 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:16:58.538518 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7193629b-f05b-48be-90bb-619c8b5b19dc None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:58.633408 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7193629b-f05b-48be-90bb-619c8b5b19dc None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:58.667301 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef288cc-fe07-4fae-82d4-0dbef28e3db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:58.667938 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef288cc-fe07-4fae-82d4-0dbef28e3db8 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:58.668214 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef288cc-fe07-4fae-82d4-0dbef28e3db8 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:58.668483 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef288cc-fe07-4fae-82d4-0dbef28e3db8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:58.669298 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef288cc-fe07-4fae-82d4-0dbef28e3db8 None admin] RBAC: Adding query filter params (role.id=4318f86ba9f74cd4a0bef1c311276657, scope.project.id=1dfa624cf8b64a7b858645cab57a1589, user.id=ca547291e8574b33856f0b453b080157) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:16:58.669605 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef288cc-fe07-4fae-82d4-0dbef28e3db8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:58.671035 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef288cc-fe07-4fae-82d4-0dbef28e3db8 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:58.677522 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 595/1185] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 15 19:16:58 2022] GET /identity/v3/role_assignments?user.id=ca547291e8574b33856f0b453b080157&scope.project.id=1dfa624cf8b64a7b858645cab57a1589&role.id=4318f86ba9f74cd4a0bef1c311276657 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:17:00.650815 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c6a59-47b7-48b0-b569-3148e28379b0 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:00.651344 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c6a59-47b7-48b0-b569-3148e28379b0 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:00.651344 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c6a59-47b7-48b0-b569-3148e28379b0 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:00.652747 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 591/1186] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:17:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:17:00.658486 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0771a84b-0a02-4adc-8f54-ed4445d493ab None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:00.658909 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0771a84b-0a02-4adc-8f54-ed4445d493ab None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:00.659044 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0771a84b-0a02-4adc-8f54-ed4445d493ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:00.675893 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-0771a84b-0a02-4adc-8f54-ed4445d493ab None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:17:00.677061 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-0771a84b-0a02-4adc-8f54-ed4445d493ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:17:00.698604 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:17:00.746157 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 596/1187] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:17:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:17:00.768144 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35c6a59-47b7-48b0-b569-3148e28379b0 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:17:00.867694 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35c6a59-47b7-48b0-b569-3148e28379b0 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:17:00.900394 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aed3f0-b5db-4db9-a826-5777e4cae96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:17:00.901223 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aed3f0-b5db-4db9-a826-5777e4cae96b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:00.901447 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aed3f0-b5db-4db9-a826-5777e4cae96b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:00.901719 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aed3f0-b5db-4db9-a826-5777e4cae96b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:00.911975 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aed3f0-b5db-4db9-a826-5777e4cae96b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:17:00.919432 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-19aed3f0-b5db-4db9-a826-5777e4cae96b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:17:00.919432 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:17:00.919432 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:17:00.919432 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:17:00.919432 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:17:00.919432 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:17:00.919432 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:17:00.919432 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:17:00.919432 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:17:00.919432 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:17:00.919432 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:17:00.919432 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:17:00.919432 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:17:00.919432 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:17:00.919432 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:17:00.919432 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:17:00.920545 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:17:00.920545 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:17:00.920545 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:17:00.920545 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:17:00.920545 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:17:00.920545 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:17:00.920545 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:17:00.920545 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:17:00.920545 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:17:00.920545 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:17:00.920545 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:17:00.920545 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:17:00.920545 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:17:00.920545 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:17:00.920545 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:17:00.920545 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:17:00.921623 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:17:00.921623 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:17:00.921623 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:17:00.921623 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:17:00.921623 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:17:00.921623 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:17:00.921623 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:17:00.921623 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:17:00.921623 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:17:00.921623 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:17:00.921623 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:17:00.921623 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:17:00.921623 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 592/1188] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jul 15 19:17:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:17:00.926790 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0771a84b-0a02-4adc-8f54-ed4445d493ab None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:17:01.022706 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0771a84b-0a02-4adc-8f54-ed4445d493ab None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:17:01.055419 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c59a88-9aca-4f69-a31a-a521aaacfd10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:17:01.056074 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c59a88-9aca-4f69-a31a-a521aaacfd10 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:01.056356 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c59a88-9aca-4f69-a31a-a521aaacfd10 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:01.056632 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c59a88-9aca-4f69-a31a-a521aaacfd10 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:01.057371 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c59a88-9aca-4f69-a31a-a521aaacfd10 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:17:01.057677 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c59a88-9aca-4f69-a31a-a521aaacfd10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:17:01.059058 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c59a88-9aca-4f69-a31a-a521aaacfd10 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:17:01.066916 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 597/1189] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 15 19:17:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:17:01.072588 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aed3f0-b5db-4db9-a826-5777e4cae96b None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:17:01.076308 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:17:01.170124 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aed3f0-b5db-4db9-a826-5777e4cae96b None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:17:01.202942 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28055344-f2fe-43fa-968c-5f7ab9e7033f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:17:01.203794 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28055344-f2fe-43fa-968c-5f7ab9e7033f None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:01.204075 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28055344-f2fe-43fa-968c-5f7ab9e7033f None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:01.204351 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28055344-f2fe-43fa-968c-5f7ab9e7033f None admin] PATH_INFO: `/v3/users/ca547291e8574b33856f0b453b080157` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:01.205531 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28055344-f2fe-43fa-968c-5f7ab9e7033f None admin] RBAC: Authorizing `identity:get_user(user_id=ca547291e8574b33856f0b453b080157)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:17:01.207002 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28055344-f2fe-43fa-968c-5f7ab9e7033f None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:17:01.208621 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 593/1190] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 15 19:17:01 2022] GET /identity/v3/users/ca547291e8574b33856f0b453b080157 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:17:01.214310 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c59a88-9aca-4f69-a31a-a521aaacfd10 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:17:01.236615 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:17:01.317334 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c59a88-9aca-4f69-a31a-a521aaacfd10 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:17:01.349891 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37407e1-5f18-40d1-8131-6b4e06a965cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:17:01.350739 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37407e1-5f18-40d1-8131-6b4e06a965cc None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:01.351024 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37407e1-5f18-40d1-8131-6b4e06a965cc None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:01.351294 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37407e1-5f18-40d1-8131-6b4e06a965cc None admin] PATH_INFO: `/v3/projects/1dfa624cf8b64a7b858645cab57a1589` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:01.352467 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37407e1-5f18-40d1-8131-6b4e06a965cc None admin] RBAC: Authorizing `identity:get_project(project_id=1dfa624cf8b64a7b858645cab57a1589)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:17:01.353909 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37407e1-5f18-40d1-8131-6b4e06a965cc None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:17:01.355609 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 598/1191] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 15 19:17:01 2022] GET /identity/v3/projects/1dfa624cf8b64a7b858645cab57a1589 => generated 361 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:17:01.361708 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28055344-f2fe-43fa-968c-5f7ab9e7033f None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:17:01.429369 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:17:01.458979 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28055344-f2fe-43fa-968c-5f7ab9e7033f None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:17:01.490352 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7236d9e-332a-48e0-9249-de33546b171a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:17:01.491298 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7236d9e-332a-48e0-9249-de33546b171a None admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:01.491575 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7236d9e-332a-48e0-9249-de33546b171a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:01.491845 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7236d9e-332a-48e0-9249-de33546b171a None admin] PATH_INFO: `/v3/projects/1dfa624cf8b64a7b858645cab57a1589/users/ca547291e8574b33856f0b453b080157/roles/4318f86ba9f74cd4a0bef1c311276657` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:01.493798 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7236d9e-332a-48e0-9249-de33546b171a None admin] RBAC: Authorizing `identity:create_grant(project_id=1dfa624cf8b64a7b858645cab57a1589, user_id=ca547291e8574b33856f0b453b080157, role_id=4318f86ba9f74cd4a0bef1c311276657)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:17:01.495211 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7236d9e-332a-48e0-9249-de33546b171a None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:17:01.565957 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 594/1192] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jul 15 19:17:01 2022] PUT /identity/v3/projects/1dfa624cf8b64a7b858645cab57a1589/users/ca547291e8574b33856f0b453b080157/roles/4318f86ba9f74cd4a0bef1c311276657 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:17:03.529837 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5775fc8f-9ebc-45a6-8d02-326a2a1bf79f None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:03.530392 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5775fc8f-9ebc-45a6-8d02-326a2a1bf79f None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:03.530392 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5775fc8f-9ebc-45a6-8d02-326a2a1bf79f None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:03.532507 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 599/1193] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:17:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:17:03.537555 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3de1aa4-583e-48b6-80f0-a736a90b252a None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:03.538010 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3de1aa4-583e-48b6-80f0-a736a90b252a None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:03.538142 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3de1aa4-583e-48b6-80f0-a736a90b252a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:03.555118 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-f3de1aa4-583e-48b6-80f0-a736a90b252a None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:17:03.556314 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-f3de1aa4-583e-48b6-80f0-a736a90b252a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:17:03.621297 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 595/1194] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:17:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:17:03.644524 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5775fc8f-9ebc-45a6-8d02-326a2a1bf79f None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:17:03.731832 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:17:03.744886 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5775fc8f-9ebc-45a6-8d02-326a2a1bf79f None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:17:03.778883 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f36a5ab-9d74-4d12-ae81-64c667ff1168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:17:03.779670 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f36a5ab-9d74-4d12-ae81-64c667ff1168 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:03.779996 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f36a5ab-9d74-4d12-ae81-64c667ff1168 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:03.780215 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f36a5ab-9d74-4d12-ae81-64c667ff1168 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:03.790616 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f36a5ab-9d74-4d12-ae81-64c667ff1168 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:17:03.798623 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-8f36a5ab-9d74-4d12-ae81-64c667ff1168 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:17:03.798623 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:17:03.798623 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:17:03.798623 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:17:03.798623 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:17:03.798623 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:17:03.798623 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:17:03.798623 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:17:03.798623 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:17:03.798623 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:17:03.798623 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:17:03.798623 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:17:03.798623 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:17:03.798623 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:17:03.798623 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:17:03.798623 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:17:03.799739 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:17:03.799739 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:17:03.799739 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:17:03.799739 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:17:03.799739 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:17:03.799739 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:17:03.799739 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:17:03.799739 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:17:03.799739 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:17:03.799739 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:17:03.799739 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:17:03.799739 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:17:03.799739 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:17:03.799739 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:17:03.799739 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:17:03.799739 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:17:03.800809 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:17:03.800809 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:17:03.800809 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:17:03.800809 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:17:03.800809 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:17:03.800809 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:17:03.800809 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:17:03.800809 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:17:03.800809 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:17:03.800809 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:17:03.800809 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:17:03.800809 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:17:03.800809 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 600/1195] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jul 15 19:17:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:17:03.806060 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3de1aa4-583e-48b6-80f0-a736a90b252a None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:17:03.865030 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:17:03.902010 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3de1aa4-583e-48b6-80f0-a736a90b252a None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:17:03.934870 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7710242-1417-4205-9b2b-32a6aa98edb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:17:03.935528 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7710242-1417-4205-9b2b-32a6aa98edb0 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:03.935808 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7710242-1417-4205-9b2b-32a6aa98edb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:03.936081 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7710242-1417-4205-9b2b-32a6aa98edb0 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:03.936822 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7710242-1417-4205-9b2b-32a6aa98edb0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:17:03.937125 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7710242-1417-4205-9b2b-32a6aa98edb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:17:03.938544 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7710242-1417-4205-9b2b-32a6aa98edb0 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:17:03.946389 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 596/1196] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 15 19:17:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:17:03.952178 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f36a5ab-9d74-4d12-ae81-64c667ff1168 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:17:04.049220 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f36a5ab-9d74-4d12-ae81-64c667ff1168 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:17:04.081427 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590b22f5-10e8-426d-8998-996ae494596c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:17:04.082312 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b22f5-10e8-426d-8998-996ae494596c None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:04.082596 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b22f5-10e8-426d-8998-996ae494596c None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:04.082873 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b22f5-10e8-426d-8998-996ae494596c None admin] PATH_INFO: `/v3/users/ca547291e8574b33856f0b453b080157` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:04.084052 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590b22f5-10e8-426d-8998-996ae494596c None admin] RBAC: Authorizing `identity:get_user(user_id=ca547291e8574b33856f0b453b080157)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:17:04.085468 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590b22f5-10e8-426d-8998-996ae494596c None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:17:04.087111 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 601/1197] 10.222.0.33 () {56 vars in 1177 bytes} [Fri Jul 15 19:17:03 2022] GET /identity/v3/users/ca547291e8574b33856f0b453b080157 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:17:04.092837 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7710242-1417-4205-9b2b-32a6aa98edb0 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:17:04.191106 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7710242-1417-4205-9b2b-32a6aa98edb0 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:17:04.212868 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:17:04.223039 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72e4d0f-0f1a-4bf6-8e68-9711e6c54777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:17:04.223680 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72e4d0f-0f1a-4bf6-8e68-9711e6c54777 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:04.223961 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72e4d0f-0f1a-4bf6-8e68-9711e6c54777 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:04.224239 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72e4d0f-0f1a-4bf6-8e68-9711e6c54777 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:04.289254 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72e4d0f-0f1a-4bf6-8e68-9711e6c54777 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:17:04.290723 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72e4d0f-0f1a-4bf6-8e68-9711e6c54777 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:17:04.386526 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 597/1198] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 15 19:17:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 19:17:04.392564 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590b22f5-10e8-426d-8998-996ae494596c None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:17:04.411298 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:17:04.489748 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590b22f5-10e8-426d-8998-996ae494596c None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:17:04.523366 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb1715d-4821-47a1-a505-36a26fae4a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:17:04.524217 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb1715d-4821-47a1-a505-36a26fae4a1f None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:04.524501 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb1715d-4821-47a1-a505-36a26fae4a1f None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:04.524778 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb1715d-4821-47a1-a505-36a26fae4a1f None admin] PATH_INFO: `/v3/projects/1dfa624cf8b64a7b858645cab57a1589` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:04.525971 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb1715d-4821-47a1-a505-36a26fae4a1f None admin] RBAC: Authorizing `identity:get_project(project_id=1dfa624cf8b64a7b858645cab57a1589)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:17:04.527747 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb1715d-4821-47a1-a505-36a26fae4a1f None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:17:04.530898 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 602/1199] 10.222.0.33 () {56 vars in 1186 bytes} [Fri Jul 15 19:17:04 2022] GET /identity/v3/projects/1dfa624cf8b64a7b858645cab57a1589 => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:17:04.537255 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72e4d0f-0f1a-4bf6-8e68-9711e6c54777 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:17:04.560789 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:17:04.634425 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72e4d0f-0f1a-4bf6-8e68-9711e6c54777 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:17:04.668173 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e4c0e7-8a99-4fd7-bd71-cbc9aaa9cd2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:17:04.668751 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e4c0e7-8a99-4fd7-bd71-cbc9aaa9cd2a None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:04.669032 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e4c0e7-8a99-4fd7-bd71-cbc9aaa9cd2a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:04.669307 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e4c0e7-8a99-4fd7-bd71-cbc9aaa9cd2a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:04.670179 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e4c0e7-8a99-4fd7-bd71-cbc9aaa9cd2a None admin] RBAC: Adding query filter params (role.id=4318f86ba9f74cd4a0bef1c311276657, scope.project.id=1dfa624cf8b64a7b858645cab57a1589, user.id=ca547291e8574b33856f0b453b080157) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:17:04.670486 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e4c0e7-8a99-4fd7-bd71-cbc9aaa9cd2a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:17:04.671943 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e4c0e7-8a99-4fd7-bd71-cbc9aaa9cd2a None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:17:04.677616 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 598/1200] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 15 19:17:04 2022] GET /identity/v3/role_assignments?user.id=ca547291e8574b33856f0b453b080157&scope.project.id=1dfa624cf8b64a7b858645cab57a1589&role.id=4318f86ba9f74cd4a0bef1c311276657 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:17:14.519250 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=58 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:17:14.675010 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=56 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:17.874154 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a117c48d-898c-4335-9e34-6de60cdcf1b9 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:17.874760 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a117c48d-898c-4335-9e34-6de60cdcf1b9 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:17.874760 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a117c48d-898c-4335-9e34-6de60cdcf1b9 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:17.876093 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 603/1201] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:18:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:18:17.881496 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3502aa7-a739-4681-b28d-fdcc53e29689 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:17.881922 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3502aa7-a739-4681-b28d-fdcc53e29689 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:17.882128 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3502aa7-a739-4681-b28d-fdcc53e29689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:17.899902 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-b3502aa7-a739-4681-b28d-fdcc53e29689 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:18:17.901134 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b3502aa7-a739-4681-b28d-fdcc53e29689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:18:17.966777 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 599/1202] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:18:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:18:17.990396 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a117c48d-898c-4335-9e34-6de60cdcf1b9 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:18.092466 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a117c48d-898c-4335-9e34-6de60cdcf1b9 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:18.128101 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffc5163-425a-4ea1-a07f-f5c2a84234fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:18.129676 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc5163-425a-4ea1-a07f-f5c2a84234fb None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:18.129676 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc5163-425a-4ea1-a07f-f5c2a84234fb None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:18.129676 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc5163-425a-4ea1-a07f-f5c2a84234fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:18.139674 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffc5163-425a-4ea1-a07f-f5c2a84234fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:18.141360 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffc5163-425a-4ea1-a07f-f5c2a84234fb None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:18.158643 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-dffc5163-425a-4ea1-a07f-f5c2a84234fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:18:18.158643 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:18:18.158643 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:18:18.158643 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:18:18.158643 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:18:18.158643 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:18:18.158643 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:18:18.158643 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:18:18.158643 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:18:18.158643 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:18:18.158643 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:18:18.158643 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:18:18.158643 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:18:18.158643 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:18:18.158643 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:18:18.159825 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:18:18.159825 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:18:18.159825 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:18:18.159825 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:18:18.159825 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:18:18.159825 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:18:18.159825 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:18:18.159825 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:18:18.159825 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:18:18.159825 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:18:18.159825 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:18:18.159825 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:18:18.159825 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:18:18.159825 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:18:18.159825 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:18:18.159825 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:18:18.159825 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:18:18.161209 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:18:18.161209 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:18:18.161209 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:18:18.161209 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:18:18.161209 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 604/1203] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:18:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:18:18.165918 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3502aa7-a739-4681-b28d-fdcc53e29689 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:18.263761 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3502aa7-a739-4681-b28d-fdcc53e29689 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:18.291942 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:18.298272 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ad182b-3c16-45b6-b65f-95e53065f3d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:18.298933 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad182b-3c16-45b6-b65f-95e53065f3d7 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:18.299215 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad182b-3c16-45b6-b65f-95e53065f3d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:18.299492 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad182b-3c16-45b6-b65f-95e53065f3d7 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:18.300213 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ad182b-3c16-45b6-b65f-95e53065f3d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:18:18.300523 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ad182b-3c16-45b6-b65f-95e53065f3d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:18.301926 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ad182b-3c16-45b6-b65f-95e53065f3d7 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:18.313472 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 600/1204] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:18:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:18:18.319327 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffc5163-425a-4ea1-a07f-f5c2a84234fb None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:18.371964 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:18.416382 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffc5163-425a-4ea1-a07f-f5c2a84234fb None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:18.450139 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f6bf09-9d68-4bad-a796-e532cfb59698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:18.450798 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f6bf09-9d68-4bad-a796-e532cfb59698 None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:18.451077 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f6bf09-9d68-4bad-a796-e532cfb59698 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:18.451352 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f6bf09-9d68-4bad-a796-e532cfb59698 None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:18.452057 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f6bf09-9d68-4bad-a796-e532cfb59698 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:18.453472 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f6bf09-9d68-4bad-a796-e532cfb59698 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:18.579604 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 605/1205] 10.222.0.33 () {60 vars in 1132 bytes} [Fri Jul 15 19:18:18 2022] POST /identity/v3/users => generated 251 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:18:20.602274 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6267c89b-aa1d-48fc-b42e-e705f5b2a439 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:20.602848 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6267c89b-aa1d-48fc-b42e-e705f5b2a439 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:20.602848 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6267c89b-aa1d-48fc-b42e-e705f5b2a439 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:20.604139 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 601/1206] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:18:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:18:20.609725 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacfca12-6120-4bde-b765-e17e03a37e3f None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:20.610171 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacfca12-6120-4bde-b765-e17e03a37e3f None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:20.610305 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacfca12-6120-4bde-b765-e17e03a37e3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:20.627101 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-dacfca12-6120-4bde-b765-e17e03a37e3f None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:18:20.628282 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-dacfca12-6120-4bde-b765-e17e03a37e3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:18:20.695694 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 606/1207] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:18:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:18:20.719300 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6267c89b-aa1d-48fc-b42e-e705f5b2a439 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:20.822566 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6267c89b-aa1d-48fc-b42e-e705f5b2a439 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:20.855098 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a7e2c7-e63b-4410-a90e-703cc8f785eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:20.856356 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a7e2c7-e63b-4410-a90e-703cc8f785eb None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:20.856636 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a7e2c7-e63b-4410-a90e-703cc8f785eb None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:20.856909 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a7e2c7-e63b-4410-a90e-703cc8f785eb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:20.867154 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a7e2c7-e63b-4410-a90e-703cc8f785eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:18:20.874924 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-24a7e2c7-e63b-4410-a90e-703cc8f785eb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:18:20.874924 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:18:20.874924 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:18:20.874924 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:18:20.874924 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:18:20.874924 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:18:20.874924 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:18:20.874924 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:18:20.874924 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:18:20.874924 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:18:20.874924 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:18:20.874924 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:18:20.874924 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:18:20.874924 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:18:20.874924 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:18:20.874924 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:18:20.884164 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:18:20.884164 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:18:20.884164 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:18:20.884164 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:18:20.884164 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:18:20.884164 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:18:20.884164 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:18:20.884164 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:18:20.884164 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:18:20.884164 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:18:20.884164 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:18:20.884164 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:18:20.884164 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:18:20.884164 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:18:20.884164 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:18:20.884164 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:18:20.885241 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:18:20.885241 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:18:20.885241 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:18:20.885241 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:18:20.885241 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:18:20.885241 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:18:20.885241 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:18:20.885241 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:18:20.885241 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:18:20.885241 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:18:20.885241 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:18:20.885241 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:18:20.885241 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 602/1208] 10.222.0.33 () {56 vars in 1102 bytes} [Fri Jul 15 19:18:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:18:20.886163 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacfca12-6120-4bde-b765-e17e03a37e3f None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:20.924757 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:20.985544 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacfca12-6120-4bde-b765-e17e03a37e3f None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:21.019743 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46cc982-c79e-4285-9574-ec8fab0a82c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:21.020406 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46cc982-c79e-4285-9574-ec8fab0a82c2 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:21.020689 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46cc982-c79e-4285-9574-ec8fab0a82c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:21.020963 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46cc982-c79e-4285-9574-ec8fab0a82c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:21.021705 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46cc982-c79e-4285-9574-ec8fab0a82c2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:18:21.022053 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46cc982-c79e-4285-9574-ec8fab0a82c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:21.023406 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46cc982-c79e-4285-9574-ec8fab0a82c2 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:21.031328 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 607/1209] 10.222.0.33 () {56 vars in 1103 bytes} [Fri Jul 15 19:18:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:18:21.037573 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a7e2c7-e63b-4410-a90e-703cc8f785eb None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:21.087260 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:21.135382 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a7e2c7-e63b-4410-a90e-703cc8f785eb None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:21.169533 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9888778b-35f4-4c4d-be7b-d46e174fb881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:21.170353 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9888778b-35f4-4c4d-be7b-d46e174fb881 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:21.170634 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9888778b-35f4-4c4d-be7b-d46e174fb881 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:21.170907 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9888778b-35f4-4c4d-be7b-d46e174fb881 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:21.181044 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9888778b-35f4-4c4d-be7b-d46e174fb881 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:21.182568 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9888778b-35f4-4c4d-be7b-d46e174fb881 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:21.200361 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-9888778b-35f4-4c4d-be7b-d46e174fb881 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:18:21.200361 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:18:21.200361 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:18:21.200361 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:18:21.200361 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:18:21.200361 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:18:21.200361 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:18:21.200361 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:18:21.200361 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:18:21.200361 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:18:21.200361 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:18:21.200361 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:18:21.200361 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:18:21.200361 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:18:21.200361 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:18:21.201447 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:18:21.201447 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:18:21.201447 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:18:21.201447 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:18:21.201447 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:18:21.201447 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:18:21.201447 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:18:21.201447 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:18:21.201447 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:18:21.201447 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:18:21.201447 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:18:21.201447 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:18:21.201447 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:18:21.201447 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:18:21.201447 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:18:21.201447 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:18:21.201447 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:18:21.206353 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:18:21.206353 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:18:21.206353 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:18:21.206353 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:18:21.206353 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 603/1210] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:18:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:18:21.207662 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46cc982-c79e-4285-9574-ec8fab0a82c2 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:21.304468 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:21.307373 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46cc982-c79e-4285-9574-ec8fab0a82c2 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:21.340161 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0329863-eeac-40bb-89d7-01b73ec19222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:21.340817 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0329863-eeac-40bb-89d7-01b73ec19222 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:21.341102 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0329863-eeac-40bb-89d7-01b73ec19222 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:21.341379 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0329863-eeac-40bb-89d7-01b73ec19222 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:21.342128 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0329863-eeac-40bb-89d7-01b73ec19222 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:18:21.342437 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0329863-eeac-40bb-89d7-01b73ec19222 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:21.343792 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0329863-eeac-40bb-89d7-01b73ec19222 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:21.354800 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 608/1211] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:18:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:18:21.361065 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9888778b-35f4-4c4d-be7b-d46e174fb881 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:21.459848 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9888778b-35f4-4c4d-be7b-d46e174fb881 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:21.461281 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:21.493286 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5828c0f-27ca-422b-bd60-4fb8bcfa3f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:21.494115 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5828c0f-27ca-422b-bd60-4fb8bcfa3f91 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:21.494400 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5828c0f-27ca-422b-bd60-4fb8bcfa3f91 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:21.494673 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5828c0f-27ca-422b-bd60-4fb8bcfa3f91 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:21.501790 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5828c0f-27ca-422b-bd60-4fb8bcfa3f91 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:21.503333 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5828c0f-27ca-422b-bd60-4fb8bcfa3f91 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:21.521845 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-c5828c0f-27ca-422b-bd60-4fb8bcfa3f91 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 15 19:18:21.521845 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:18:21.521845 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:18:21.521845 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:18:21.521845 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:18:21.521845 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:18:21.521845 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:18:21.521845 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:18:21.521845 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:18:21.521845 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:18:21.521845 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:18:21.521845 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:18:21.521845 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 19:18:21.521845 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 19:18:21.521845 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 19:18:21.521845 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 19:18:21.523076 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:18:21.523076 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:18:21.523076 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:18:21.523076 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:18:21.523076 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:18:21.523076 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:18:21.523076 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:18:21.523076 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:18:21.523076 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:18:21.523076 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:18:21.523076 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:18:21.523076 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:18:21.523076 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:18:21.523076 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:18:21.523076 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:18:21.523076 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:18:21.523076 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:18:21.523076 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:18:21.524113 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:18:21.524113 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:18:21.524113 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 19:18:21.524113 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 19:18:21.524113 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 19:18:21.524113 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 19:18:21.524113 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 19:18:21.524113 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 19:18:21.524113 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 15 19:18:21.524113 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:18:21.524113 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 604/1212] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:18:21 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:18:21.530574 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0329863-eeac-40bb-89d7-01b73ec19222 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:21.633921 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0329863-eeac-40bb-89d7-01b73ec19222 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:21.667608 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf352c2b-8dbf-41af-a18f-85caab4076c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:21.668819 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf352c2b-8dbf-41af-a18f-85caab4076c9 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:21.669095 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf352c2b-8dbf-41af-a18f-85caab4076c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:21.669365 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf352c2b-8dbf-41af-a18f-85caab4076c9 None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:21.670535 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf352c2b-8dbf-41af-a18f-85caab4076c9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:18:21.670535 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf352c2b-8dbf-41af-a18f-85caab4076c9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:21.674701 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf352c2b-8dbf-41af-a18f-85caab4076c9 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:21.688790 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 609/1213] 10.222.0.33 () {56 vars in 1143 bytes} [Fri Jul 15 19:18:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:18:21.694599 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5828c0f-27ca-422b-bd60-4fb8bcfa3f91 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:21.793721 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5828c0f-27ca-422b-bd60-4fb8bcfa3f91 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:21.826676 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d76323-bdda-4c64-955c-96b9f6a55a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:21.827316 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d76323-bdda-4c64-955c-96b9f6a55a5d None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:21.827591 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d76323-bdda-4c64-955c-96b9f6a55a5d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:21.827858 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d76323-bdda-4c64-955c-96b9f6a55a5d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:21.857328 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:21.895099 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d76323-bdda-4c64-955c-96b9f6a55a5d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:21.896530 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d76323-bdda-4c64-955c-96b9f6a55a5d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:21.997025 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 605/1214] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 15 19:18:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 19:18:22.002759 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf352c2b-8dbf-41af-a18f-85caab4076c9 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:22.021866 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:22.099455 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf352c2b-8dbf-41af-a18f-85caab4076c9 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:22.131397 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db838c75-a738-42b0-a0c4-c7e0276b84e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:22.132190 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db838c75-a738-42b0-a0c4-c7e0276b84e4 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:22.132468 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db838c75-a738-42b0-a0c4-c7e0276b84e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:22.132757 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db838c75-a738-42b0-a0c4-c7e0276b84e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:22.143626 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db838c75-a738-42b0-a0c4-c7e0276b84e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:22.145058 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db838c75-a738-42b0-a0c4-c7e0276b84e4 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:22.164284 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-db838c75-a738-42b0-a0c4-c7e0276b84e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:18:22.164284 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:18:22.164284 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:18:22.164284 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:18:22.164284 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:18:22.164284 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:18:22.164284 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:18:22.164284 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:18:22.164284 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:18:22.164284 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:18:22.164284 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:18:22.164284 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:18:22.164284 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:18:22.164284 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:18:22.164284 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:18:22.165438 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:18:22.165438 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:18:22.165438 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:18:22.165438 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:18:22.165438 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:18:22.165438 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:18:22.165438 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:18:22.165438 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:18:22.165438 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:18:22.165438 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:18:22.165438 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:18:22.165438 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:18:22.165438 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:18:22.165438 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:18:22.165438 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:18:22.165438 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:18:22.165438 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:18:22.166453 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:18:22.166453 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:18:22.166453 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:18:22.166453 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:18:22.166453 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 610/1215] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:18:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:18:22.171449 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d76323-bdda-4c64-955c-96b9f6a55a5d None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:22.230684 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:22.268315 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d76323-bdda-4c64-955c-96b9f6a55a5d None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:22.301121 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f518397b-e414-426d-99a2-8b5b608314a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:22.301845 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f518397b-e414-426d-99a2-8b5b608314a4 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:22.302131 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f518397b-e414-426d-99a2-8b5b608314a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:22.302411 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f518397b-e414-426d-99a2-8b5b608314a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:22.303133 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f518397b-e414-426d-99a2-8b5b608314a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:18:22.303438 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f518397b-e414-426d-99a2-8b5b608314a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:22.304840 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f518397b-e414-426d-99a2-8b5b608314a4 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:22.316426 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 606/1216] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:18:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:18:22.322190 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db838c75-a738-42b0-a0c4-c7e0276b84e4 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:22.391077 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:22.419473 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db838c75-a738-42b0-a0c4-c7e0276b84e4 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:22.452136 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f827cd-35ca-47c0-bf9d-5592a9265334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:22.452942 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f827cd-35ca-47c0-bf9d-5592a9265334 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:22.453224 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f827cd-35ca-47c0-bf9d-5592a9265334 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:22.453499 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f827cd-35ca-47c0-bf9d-5592a9265334 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:22.459891 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f827cd-35ca-47c0-bf9d-5592a9265334 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:22.461316 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f827cd-35ca-47c0-bf9d-5592a9265334 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:22.474584 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-98f827cd-35ca-47c0-bf9d-5592a9265334 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:18:22.474584 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:18:22.474584 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:18:22.474584 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:18:22.474584 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:18:22.474584 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:18:22.474584 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:18:22.474584 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:18:22.474584 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:18:22.474584 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:18:22.474584 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:18:22.474584 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:18:22.474584 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 19:18:22.474584 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 19:18:22.474584 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 19:18:22.475662 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 19:18:22.475662 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:18:22.475662 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:18:22.475662 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:18:22.475662 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:18:22.475662 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:18:22.475662 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:18:22.475662 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:18:22.475662 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:18:22.475662 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:18:22.475662 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:18:22.475662 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:18:22.475662 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:18:22.475662 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:18:22.475662 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:18:22.475662 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:18:22.475662 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:18:22.476652 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 19:18:22.476652 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 19:18:22.476652 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:18:22.476652 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:18:22.476652 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:18:22.476652 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:18:22.476652 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:18:22.476652 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:18:22.476652 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 611/1217] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 15 19:18:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:18:22.481566 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f518397b-e414-426d-99a2-8b5b608314a4 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:22.579375 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f518397b-e414-426d-99a2-8b5b608314a4 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:22.614810 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa9a21c-2b3f-40d8-8d52-2c15db9c9e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:22.617019 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa9a21c-2b3f-40d8-8d52-2c15db9c9e22 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:22.617019 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa9a21c-2b3f-40d8-8d52-2c15db9c9e22 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:22.617019 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa9a21c-2b3f-40d8-8d52-2c15db9c9e22 None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:22.617019 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa9a21c-2b3f-40d8-8d52-2c15db9c9e22 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:18:22.617378 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa9a21c-2b3f-40d8-8d52-2c15db9c9e22 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:22.618555 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa9a21c-2b3f-40d8-8d52-2c15db9c9e22 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:22.631748 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 607/1218] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jul 15 19:18:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:18:22.638028 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f827cd-35ca-47c0-bf9d-5592a9265334 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:22.666639 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:22.734194 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f827cd-35ca-47c0-bf9d-5592a9265334 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:22.766463 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdf2520-09ed-4e5b-906b-c7a9d6311a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:22.767104 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdf2520-09ed-4e5b-906b-c7a9d6311a1e None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:22.767383 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdf2520-09ed-4e5b-906b-c7a9d6311a1e None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:22.767654 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdf2520-09ed-4e5b-906b-c7a9d6311a1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:22.768468 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdf2520-09ed-4e5b-906b-c7a9d6311a1e None admin] RBAC: Adding query filter params (role.id=70de044d6962417593b6d03571e7502e, scope.project.id=55a756ccaef14cb49b26b0d68dd0b466, user.id=52a0fce231ba4785a685ac9898d07503) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:18:22.768778 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdf2520-09ed-4e5b-906b-c7a9d6311a1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:22.770212 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdf2520-09ed-4e5b-906b-c7a9d6311a1e None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:22.775557 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 612/1219] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 15 19:18:22 2022] GET /identity/v3/role_assignments?user.id=52a0fce231ba4785a685ac9898d07503&scope.project.id=55a756ccaef14cb49b26b0d68dd0b466&role.id=70de044d6962417593b6d03571e7502e => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:18:22.821252 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:24.786048 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9d739-852d-46ce-8797-4a281f42fd2b None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:24.786598 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9d739-852d-46ce-8797-4a281f42fd2b None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:24.786598 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9d739-852d-46ce-8797-4a281f42fd2b None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:24.787981 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 608/1220] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:18:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:18:24.793884 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158f6691-e643-4854-8f41-cb98062ae824 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:24.794316 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158f6691-e643-4854-8f41-cb98062ae824 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:24.794784 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158f6691-e643-4854-8f41-cb98062ae824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:24.813904 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-158f6691-e643-4854-8f41-cb98062ae824 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:18:24.820106 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-158f6691-e643-4854-8f41-cb98062ae824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:18:24.883954 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 613/1221] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:18:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:18:24.907501 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c9d739-852d-46ce-8797-4a281f42fd2b None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:25.015438 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c9d739-852d-46ce-8797-4a281f42fd2b None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:25.055919 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dbf586-0932-42f2-a42a-2f35b580fe32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:25.056727 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dbf586-0932-42f2-a42a-2f35b580fe32 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:25.057010 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dbf586-0932-42f2-a42a-2f35b580fe32 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:25.057280 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dbf586-0932-42f2-a42a-2f35b580fe32 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:25.068411 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dbf586-0932-42f2-a42a-2f35b580fe32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:18:25.076486 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-51dbf586-0932-42f2-a42a-2f35b580fe32 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:18:25.076486 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:18:25.076486 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:18:25.076486 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:18:25.076486 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:18:25.076486 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:18:25.076486 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:18:25.076486 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:18:25.076486 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:18:25.076486 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:18:25.076486 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:18:25.076486 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:18:25.076486 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:18:25.076486 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:18:25.076486 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:18:25.076486 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:18:25.077683 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:18:25.077683 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:18:25.077683 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:18:25.077683 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:18:25.077683 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:18:25.077683 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:18:25.077683 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:18:25.077683 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:18:25.077683 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:18:25.077683 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:18:25.077683 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:18:25.077683 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:18:25.077683 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:18:25.077683 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:18:25.077683 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:18:25.077683 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:18:25.079000 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:18:25.079000 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:18:25.079000 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:18:25.079000 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:18:25.079000 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:18:25.079000 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:18:25.079000 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:18:25.079000 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:18:25.079000 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:18:25.079000 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:18:25.079000 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:18:25.079000 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:18:25.079000 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 609/1222] 10.222.0.33 () {56 vars in 1102 bytes} [Fri Jul 15 19:18:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:18:25.083955 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158f6691-e643-4854-8f41-cb98062ae824 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:25.191252 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158f6691-e643-4854-8f41-cb98062ae824 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:25.218109 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:25.229914 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394dcd75-9a8f-4a56-8a7c-952b2f960613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:25.231158 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394dcd75-9a8f-4a56-8a7c-952b2f960613 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:25.231521 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394dcd75-9a8f-4a56-8a7c-952b2f960613 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:25.231710 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394dcd75-9a8f-4a56-8a7c-952b2f960613 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:25.232464 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394dcd75-9a8f-4a56-8a7c-952b2f960613 None admin] RBAC: Adding query filter params (name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:18:25.232768 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394dcd75-9a8f-4a56-8a7c-952b2f960613 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:25.234207 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394dcd75-9a8f-4a56-8a7c-952b2f960613 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:25.242455 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 614/1223] 10.222.0.33 () {56 vars in 1103 bytes} [Fri Jul 15 19:18:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:18:25.248427 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dbf586-0932-42f2-a42a-2f35b580fe32 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:25.295562 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:25.352049 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dbf586-0932-42f2-a42a-2f35b580fe32 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:25.388175 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4a71e2-d3bc-489a-8fc1-0961e9879b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:25.389505 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a71e2-d3bc-489a-8fc1-0961e9879b62 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:25.390305 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a71e2-d3bc-489a-8fc1-0961e9879b62 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:25.390814 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a71e2-d3bc-489a-8fc1-0961e9879b62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:25.401697 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4a71e2-d3bc-489a-8fc1-0961e9879b62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:25.403719 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4a71e2-d3bc-489a-8fc1-0961e9879b62 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:25.420865 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-ba4a71e2-d3bc-489a-8fc1-0961e9879b62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:18:25.420865 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:18:25.420865 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:18:25.420865 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:18:25.420865 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:18:25.420865 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:18:25.420865 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:18:25.420865 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:18:25.420865 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:18:25.420865 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:18:25.420865 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:18:25.420865 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:18:25.420865 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:18:25.420865 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:18:25.420865 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:18:25.422074 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:18:25.422074 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:18:25.422074 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:18:25.422074 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:18:25.422074 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:18:25.422074 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:18:25.422074 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:18:25.422074 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:18:25.422074 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:18:25.422074 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:18:25.422074 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:18:25.422074 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:18:25.422074 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:18:25.422074 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:18:25.422074 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:18:25.422074 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:18:25.422074 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:18:25.423305 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:18:25.423305 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:18:25.423305 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:18:25.423305 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:18:25.423305 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 610/1224] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:18:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:18:25.428736 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394dcd75-9a8f-4a56-8a7c-952b2f960613 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:25.529134 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394dcd75-9a8f-4a56-8a7c-952b2f960613 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:25.564347 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196f331f-af00-4bbf-82a9-9edc11516d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:25.565397 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196f331f-af00-4bbf-82a9-9edc11516d6c None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:25.565681 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196f331f-af00-4bbf-82a9-9edc11516d6c None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:25.565978 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196f331f-af00-4bbf-82a9-9edc11516d6c None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:25.566698 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196f331f-af00-4bbf-82a9-9edc11516d6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:18:25.567004 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196f331f-af00-4bbf-82a9-9edc11516d6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:25.568367 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196f331f-af00-4bbf-82a9-9edc11516d6c None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:25.583892 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 615/1225] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:18:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:18:25.589578 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4a71e2-d3bc-489a-8fc1-0961e9879b62 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:25.680802 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:25.687275 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4a71e2-d3bc-489a-8fc1-0961e9879b62 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:25.719971 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159140ef-a6bb-4a91-976b-d2d1f3945664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:25.720765 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159140ef-a6bb-4a91-976b-d2d1f3945664 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:25.721040 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159140ef-a6bb-4a91-976b-d2d1f3945664 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:25.721308 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159140ef-a6bb-4a91-976b-d2d1f3945664 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:25.728407 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159140ef-a6bb-4a91-976b-d2d1f3945664 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:25.729933 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159140ef-a6bb-4a91-976b-d2d1f3945664 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:25.743662 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-159140ef-a6bb-4a91-976b-d2d1f3945664 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 15 19:18:25.743662 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:18:25.743662 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:18:25.743662 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:18:25.743662 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:18:25.743662 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:18:25.743662 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:18:25.743662 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:18:25.743662 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:18:25.743662 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:18:25.743662 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:18:25.743662 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:18:25.743662 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 19:18:25.743662 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 19:18:25.743662 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 19:18:25.743662 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 19:18:25.744791 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:18:25.744791 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:18:25.744791 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:18:25.744791 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:18:25.744791 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:18:25.744791 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:18:25.744791 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:18:25.744791 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:18:25.744791 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:18:25.744791 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:18:25.744791 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:18:25.744791 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:18:25.744791 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:18:25.744791 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:18:25.744791 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:18:25.744791 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:18:25.744791 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:18:25.744791 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:18:25.746025 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:18:25.746025 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:18:25.746025 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 19:18:25.746025 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 19:18:25.746025 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 19:18:25.746025 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 19:18:25.746025 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 19:18:25.746025 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 19:18:25.746025 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 15 19:18:25.746025 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:18:25.746025 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 611/1226] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:18:25 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:18:25.750864 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196f331f-af00-4bbf-82a9-9edc11516d6c None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:25.797922 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:25.849587 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196f331f-af00-4bbf-82a9-9edc11516d6c None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:25.882294 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaa2029-5e6b-488a-8e2c-2269480fd820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:25.882956 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa2029-5e6b-488a-8e2c-2269480fd820 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:25.883238 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa2029-5e6b-488a-8e2c-2269480fd820 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:25.883519 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa2029-5e6b-488a-8e2c-2269480fd820 None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:25.884345 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaa2029-5e6b-488a-8e2c-2269480fd820 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:18:25.884653 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaa2029-5e6b-488a-8e2c-2269480fd820 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:25.886094 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaa2029-5e6b-488a-8e2c-2269480fd820 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:25.903467 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 616/1227] 10.222.0.33 () {56 vars in 1143 bytes} [Fri Jul 15 19:18:25 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:18:25.909354 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159140ef-a6bb-4a91-976b-d2d1f3945664 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:26.007767 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159140ef-a6bb-4a91-976b-d2d1f3945664 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:26.041946 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244b947c-d4fe-417d-8506-5e0d7ee2a288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:26.042744 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244b947c-d4fe-417d-8506-5e0d7ee2a288 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:26.043028 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244b947c-d4fe-417d-8506-5e0d7ee2a288 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:26.043306 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244b947c-d4fe-417d-8506-5e0d7ee2a288 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:26.053729 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244b947c-d4fe-417d-8506-5e0d7ee2a288 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:26.055233 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244b947c-d4fe-417d-8506-5e0d7ee2a288 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:26.071842 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-244b947c-d4fe-417d-8506-5e0d7ee2a288 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:18:26.071842 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:18:26.071842 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:18:26.071842 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:18:26.071842 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:18:26.071842 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:18:26.071842 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:18:26.071842 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:18:26.071842 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:18:26.071842 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:18:26.071842 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:18:26.071842 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:18:26.071842 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:18:26.071842 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:18:26.071842 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:18:26.075790 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:18:26.075790 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:18:26.075790 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:18:26.075790 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:18:26.075790 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:18:26.075790 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:18:26.075790 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:18:26.075790 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:18:26.075790 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:18:26.075790 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:18:26.075790 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:18:26.075790 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:18:26.075790 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:18:26.075790 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:18:26.075790 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:18:26.075790 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:18:26.075790 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:18:26.076923 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:18:26.076923 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:18:26.076923 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:18:26.076923 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:18:26.076923 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 612/1228] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:18:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:18:26.079536 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaa2029-5e6b-488a-8e2c-2269480fd820 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:26.178366 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaa2029-5e6b-488a-8e2c-2269480fd820 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:26.180944 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:26.211465 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e26f8a9-2409-44df-9fb3-24196527a052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:26.212123 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e26f8a9-2409-44df-9fb3-24196527a052 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:26.212471 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e26f8a9-2409-44df-9fb3-24196527a052 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:26.212676 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e26f8a9-2409-44df-9fb3-24196527a052 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:26.213389 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e26f8a9-2409-44df-9fb3-24196527a052 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:18:26.213690 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e26f8a9-2409-44df-9fb3-24196527a052 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:26.215656 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e26f8a9-2409-44df-9fb3-24196527a052 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:26.226871 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 617/1229] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:18:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:18:26.232652 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244b947c-d4fe-417d-8506-5e0d7ee2a288 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:26.242747 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:26.333730 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244b947c-d4fe-417d-8506-5e0d7ee2a288 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:26.368504 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b944bd6-1be2-479a-99f3-2d5dc2bd86c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:26.369289 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b944bd6-1be2-479a-99f3-2d5dc2bd86c8 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:26.369567 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b944bd6-1be2-479a-99f3-2d5dc2bd86c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:26.369864 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b944bd6-1be2-479a-99f3-2d5dc2bd86c8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:26.376778 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b944bd6-1be2-479a-99f3-2d5dc2bd86c8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:26.378253 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b944bd6-1be2-479a-99f3-2d5dc2bd86c8 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:26.392774 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-8b944bd6-1be2-479a-99f3-2d5dc2bd86c8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:18:26.392774 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:18:26.392774 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:18:26.392774 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:18:26.392774 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:18:26.392774 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:18:26.392774 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:18:26.392774 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:18:26.392774 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:18:26.392774 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:18:26.392774 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:18:26.392774 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:18:26.392774 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 19:18:26.392774 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 19:18:26.392774 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 19:18:26.393888 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 19:18:26.393888 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:18:26.393888 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:18:26.393888 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:18:26.393888 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:18:26.393888 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:18:26.393888 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:18:26.393888 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:18:26.393888 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:18:26.393888 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:18:26.393888 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:18:26.393888 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:18:26.393888 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:18:26.393888 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:18:26.393888 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:18:26.393888 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:18:26.393888 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:18:26.394968 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 19:18:26.394968 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 19:18:26.394968 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:18:26.394968 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:18:26.394968 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:18:26.394968 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:18:26.394968 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:18:26.394968 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:18:26.394968 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 613/1230] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 15 19:18:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:18:26.399841 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e26f8a9-2409-44df-9fb3-24196527a052 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:26.497093 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e26f8a9-2409-44df-9fb3-24196527a052 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:26.530594 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568cdf8e-4441-42c9-9f41-02e489fbda26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:26.539139 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568cdf8e-4441-42c9-9f41-02e489fbda26 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:26.539139 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568cdf8e-4441-42c9-9f41-02e489fbda26 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:26.539139 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568cdf8e-4441-42c9-9f41-02e489fbda26 None admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:26.539139 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568cdf8e-4441-42c9-9f41-02e489fbda26 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:18:26.539139 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568cdf8e-4441-42c9-9f41-02e489fbda26 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:26.539139 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568cdf8e-4441-42c9-9f41-02e489fbda26 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:26.546804 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 618/1231] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jul 15 19:18:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:18:26.552857 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b944bd6-1be2-479a-99f3-2d5dc2bd86c8 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:26.644858 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:26.653244 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b944bd6-1be2-479a-99f3-2d5dc2bd86c8 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:26.686837 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf581752-0a8e-4e2a-baa3-f224770a6dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:26.687769 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf581752-0a8e-4e2a-baa3-f224770a6dc9 None admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:26.688040 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf581752-0a8e-4e2a-baa3-f224770a6dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:26.688308 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf581752-0a8e-4e2a-baa3-f224770a6dc9 None admin] PATH_INFO: `/v3/projects/55a756ccaef14cb49b26b0d68dd0b466/users/52a0fce231ba4785a685ac9898d07503/roles/70de044d6962417593b6d03571e7502e` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:26.703067 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf581752-0a8e-4e2a-baa3-f224770a6dc9 None admin] RBAC: Authorizing `identity:create_grant(project_id=55a756ccaef14cb49b26b0d68dd0b466, user_id=52a0fce231ba4785a685ac9898d07503, role_id=70de044d6962417593b6d03571e7502e)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:26.704622 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf581752-0a8e-4e2a-baa3-f224770a6dc9 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:26.826107 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 614/1232] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jul 15 19:18:26 2022] PUT /identity/v3/projects/55a756ccaef14cb49b26b0d68dd0b466/users/52a0fce231ba4785a685ac9898d07503/roles/70de044d6962417593b6d03571e7502e => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:18:28.818960 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8dd9a4-b878-4285-9564-82a9b0d1e454 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:28.819472 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8dd9a4-b878-4285-9564-82a9b0d1e454 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:28.819570 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8dd9a4-b878-4285-9564-82a9b0d1e454 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:28.820873 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 619/1233] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:18:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:18:28.826372 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9383ca35-0bce-4d18-89e7-c877a4fe2f14 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:28.826953 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9383ca35-0bce-4d18-89e7-c877a4fe2f14 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:28.826953 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9383ca35-0bce-4d18-89e7-c877a4fe2f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:28.843552 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-9383ca35-0bce-4d18-89e7-c877a4fe2f14 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:18:28.844740 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9383ca35-0bce-4d18-89e7-c877a4fe2f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:18:28.909886 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 615/1234] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:18:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:18:28.933696 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8dd9a4-b878-4285-9564-82a9b0d1e454 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:28.958013 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:29.032390 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8dd9a4-b878-4285-9564-82a9b0d1e454 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:29.064641 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9c8002-f2ae-400e-bdca-b6bbe7dfc098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:29.065381 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9c8002-f2ae-400e-bdca-b6bbe7dfc098 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:29.065664 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9c8002-f2ae-400e-bdca-b6bbe7dfc098 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:29.065965 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9c8002-f2ae-400e-bdca-b6bbe7dfc098 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:29.076521 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9c8002-f2ae-400e-bdca-b6bbe7dfc098 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:18:29.084127 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-7d9c8002-f2ae-400e-bdca-b6bbe7dfc098 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:18:29.084127 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:18:29.084127 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:18:29.084127 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:18:29.084127 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:18:29.084127 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:18:29.084127 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:18:29.084127 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:18:29.084127 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:18:29.084127 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:18:29.084127 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:18:29.084127 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:18:29.084127 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:18:29.084127 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:18:29.084127 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:18:29.084127 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:18:29.086250 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:18:29.086250 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:18:29.086250 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:18:29.086250 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:18:29.086250 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:18:29.086250 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:18:29.086250 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:18:29.086250 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:18:29.086250 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:18:29.086250 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:18:29.086250 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:18:29.086250 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:18:29.086250 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:18:29.086250 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:18:29.086250 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:18:29.086250 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:18:29.087420 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:18:29.087420 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:18:29.087420 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:18:29.087420 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:18:29.087420 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:18:29.087420 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:18:29.087420 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:18:29.087420 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:18:29.087420 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:18:29.087420 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:18:29.087420 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:18:29.087420 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:18:29.087420 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 620/1235] 10.222.0.33 () {56 vars in 1102 bytes} [Fri Jul 15 19:18:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:18:29.091880 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9383ca35-0bce-4d18-89e7-c877a4fe2f14 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:29.136953 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:29.188682 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9383ca35-0bce-4d18-89e7-c877a4fe2f14 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:29.222055 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b048af8-36ba-48d3-8ad4-f70614251ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:29.222718 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b048af8-36ba-48d3-8ad4-f70614251ec4 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:29.223002 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b048af8-36ba-48d3-8ad4-f70614251ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:29.223279 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b048af8-36ba-48d3-8ad4-f70614251ec4 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:29.224414 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b048af8-36ba-48d3-8ad4-f70614251ec4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:18:29.224841 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b048af8-36ba-48d3-8ad4-f70614251ec4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:29.226649 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b048af8-36ba-48d3-8ad4-f70614251ec4 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:29.235326 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 616/1236] 10.222.0.33 () {56 vars in 1103 bytes} [Fri Jul 15 19:18:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:18:29.241039 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9c8002-f2ae-400e-bdca-b6bbe7dfc098 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:29.324901 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:29.340162 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9c8002-f2ae-400e-bdca-b6bbe7dfc098 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:29.374047 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d56b81-1df5-4c37-b2e8-be19b17a8f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:29.374832 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d56b81-1df5-4c37-b2e8-be19b17a8f39 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:29.375111 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d56b81-1df5-4c37-b2e8-be19b17a8f39 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:29.375382 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d56b81-1df5-4c37-b2e8-be19b17a8f39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:29.388110 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d56b81-1df5-4c37-b2e8-be19b17a8f39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:29.388110 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d56b81-1df5-4c37-b2e8-be19b17a8f39 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:29.405848 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-44d56b81-1df5-4c37-b2e8-be19b17a8f39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:18:29.405848 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:18:29.405848 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:18:29.405848 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:18:29.405848 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:18:29.405848 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:18:29.405848 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:18:29.405848 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:18:29.405848 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:18:29.405848 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:18:29.405848 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:18:29.405848 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:18:29.405848 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:18:29.405848 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:18:29.405848 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:18:29.406955 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:18:29.406955 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:18:29.406955 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:18:29.406955 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:18:29.406955 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:18:29.406955 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:18:29.406955 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:18:29.406955 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:18:29.406955 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:18:29.406955 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:18:29.406955 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:18:29.406955 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:18:29.406955 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:18:29.406955 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:18:29.406955 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:18:29.406955 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:18:29.406955 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:18:29.408004 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:18:29.408004 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:18:29.408004 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:18:29.408004 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:18:29.408004 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 621/1237] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:18:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:18:29.413082 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b048af8-36ba-48d3-8ad4-f70614251ec4 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:29.510662 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b048af8-36ba-48d3-8ad4-f70614251ec4 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:29.520364 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:29.544842 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12bf55f-b4de-4da9-8db6-26abfc5c0ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:29.545516 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12bf55f-b4de-4da9-8db6-26abfc5c0ec1 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:29.545830 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12bf55f-b4de-4da9-8db6-26abfc5c0ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:29.546112 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12bf55f-b4de-4da9-8db6-26abfc5c0ec1 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:29.546849 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12bf55f-b4de-4da9-8db6-26abfc5c0ec1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:18:29.547158 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12bf55f-b4de-4da9-8db6-26abfc5c0ec1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:29.548530 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12bf55f-b4de-4da9-8db6-26abfc5c0ec1 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:29.560319 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 617/1238] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:18:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:18:29.565944 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d56b81-1df5-4c37-b2e8-be19b17a8f39 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:29.675087 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d56b81-1df5-4c37-b2e8-be19b17a8f39 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:29.718450 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21407b87-32cd-477c-bbf4-c219c6ce9c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:29.719250 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21407b87-32cd-477c-bbf4-c219c6ce9c22 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:29.719526 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21407b87-32cd-477c-bbf4-c219c6ce9c22 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:29.719797 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21407b87-32cd-477c-bbf4-c219c6ce9c22 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:29.722808 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:29.728261 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21407b87-32cd-477c-bbf4-c219c6ce9c22 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:29.730715 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21407b87-32cd-477c-bbf4-c219c6ce9c22 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:29.749089 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-21407b87-32cd-477c-bbf4-c219c6ce9c22 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 15 19:18:29.749089 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:18:29.749089 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:18:29.749089 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:18:29.749089 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:18:29.749089 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:18:29.749089 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:18:29.749089 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:18:29.749089 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:18:29.749089 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:18:29.749089 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:18:29.749089 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:18:29.749089 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 19:18:29.749089 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 19:18:29.749089 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 19:18:29.749089 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 19:18:29.751885 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:18:29.751885 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:18:29.751885 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:18:29.751885 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:18:29.751885 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:18:29.751885 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:18:29.751885 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:18:29.751885 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:18:29.751885 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:18:29.751885 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:18:29.751885 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:18:29.751885 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:18:29.751885 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:18:29.751885 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:18:29.751885 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:18:29.751885 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:18:29.751885 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:18:29.751885 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:18:29.753310 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:18:29.753310 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:18:29.753310 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 19:18:29.753310 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 19:18:29.753310 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 19:18:29.753310 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 19:18:29.753310 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 19:18:29.753310 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 19:18:29.753310 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 15 19:18:29.753310 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:18:29.753310 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 622/1239] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:18:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:18:29.756276 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12bf55f-b4de-4da9-8db6-26abfc5c0ec1 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:29.857379 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12bf55f-b4de-4da9-8db6-26abfc5c0ec1 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:29.891517 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3d7167-caea-4bc2-a458-6fcd17862180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:29.892183 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3d7167-caea-4bc2-a458-6fcd17862180 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:29.892465 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3d7167-caea-4bc2-a458-6fcd17862180 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:29.892741 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3d7167-caea-4bc2-a458-6fcd17862180 None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:29.893617 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3d7167-caea-4bc2-a458-6fcd17862180 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:18:29.893956 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3d7167-caea-4bc2-a458-6fcd17862180 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:29.895472 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3d7167-caea-4bc2-a458-6fcd17862180 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:29.912770 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 618/1240] 10.222.0.33 () {56 vars in 1143 bytes} [Fri Jul 15 19:18:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:18:29.918657 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21407b87-32cd-477c-bbf4-c219c6ce9c22 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:30.016971 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21407b87-32cd-477c-bbf4-c219c6ce9c22 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:30.051635 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e880aa-ff25-4927-9876-0114efb35246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:30.052290 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e880aa-ff25-4927-9876-0114efb35246 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:30.052570 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e880aa-ff25-4927-9876-0114efb35246 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:30.052841 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e880aa-ff25-4927-9876-0114efb35246 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:30.121908 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:30.128326 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e880aa-ff25-4927-9876-0114efb35246 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:30.129777 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e880aa-ff25-4927-9876-0114efb35246 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:30.223707 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 623/1241] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 15 19:18:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 19:18:30.229519 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3d7167-caea-4bc2-a458-6fcd17862180 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:30.256767 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:30.327224 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3d7167-caea-4bc2-a458-6fcd17862180 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:30.359530 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e809513-73aa-4513-873a-8086a62be1c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:30.360317 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e809513-73aa-4513-873a-8086a62be1c7 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:30.360592 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e809513-73aa-4513-873a-8086a62be1c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:30.360868 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e809513-73aa-4513-873a-8086a62be1c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:30.371179 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e809513-73aa-4513-873a-8086a62be1c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:30.372604 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e809513-73aa-4513-873a-8086a62be1c7 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:30.390985 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-5e809513-73aa-4513-873a-8086a62be1c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:18:30.390985 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:18:30.390985 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:18:30.390985 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:18:30.390985 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:18:30.390985 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:18:30.390985 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:18:30.390985 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:18:30.390985 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:18:30.390985 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:18:30.390985 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:18:30.390985 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:18:30.390985 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:18:30.390985 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:18:30.390985 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:18:30.392075 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:18:30.392075 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:18:30.392075 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:18:30.392075 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:18:30.392075 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:18:30.392075 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:18:30.392075 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:18:30.392075 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:18:30.392075 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:18:30.392075 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:18:30.392075 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:18:30.392075 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:18:30.392075 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:18:30.392075 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:18:30.392075 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:18:30.392075 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:18:30.392075 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:18:30.393224 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:18:30.393224 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:18:30.393224 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:18:30.393224 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:18:30.393224 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 619/1242] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:18:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:18:30.398016 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e880aa-ff25-4927-9876-0114efb35246 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:30.490783 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:30.496756 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e880aa-ff25-4927-9876-0114efb35246 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:30.529772 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd4f837-0b2f-4f6e-94fa-83268c931c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:30.533588 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd4f837-0b2f-4f6e-94fa-83268c931c42 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:30.533588 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd4f837-0b2f-4f6e-94fa-83268c931c42 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:30.533588 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd4f837-0b2f-4f6e-94fa-83268c931c42 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:30.533588 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd4f837-0b2f-4f6e-94fa-83268c931c42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:18:30.533588 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd4f837-0b2f-4f6e-94fa-83268c931c42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:30.533588 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd4f837-0b2f-4f6e-94fa-83268c931c42 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:30.544936 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 624/1243] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:18:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:18:30.550947 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e809513-73aa-4513-873a-8086a62be1c7 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:30.628606 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:30.649157 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e809513-73aa-4513-873a-8086a62be1c7 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:30.682600 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce59a7a3-6a7d-45a3-88ff-a46564698c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:30.683968 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce59a7a3-6a7d-45a3-88ff-a46564698c24 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:30.684248 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce59a7a3-6a7d-45a3-88ff-a46564698c24 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:30.684522 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce59a7a3-6a7d-45a3-88ff-a46564698c24 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:30.691261 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce59a7a3-6a7d-45a3-88ff-a46564698c24 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:30.692760 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce59a7a3-6a7d-45a3-88ff-a46564698c24 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:30.706227 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-ce59a7a3-6a7d-45a3-88ff-a46564698c24 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:18:30.706227 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:18:30.706227 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:18:30.706227 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:18:30.706227 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:18:30.706227 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:18:30.706227 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:18:30.706227 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:18:30.706227 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:18:30.706227 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:18:30.706227 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:18:30.706227 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:18:30.706227 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 19:18:30.706227 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 19:18:30.706227 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 19:18:30.707308 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 19:18:30.707308 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:18:30.707308 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:18:30.707308 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:18:30.707308 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:18:30.707308 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:18:30.707308 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:18:30.707308 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:18:30.707308 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:18:30.707308 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:18:30.707308 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:18:30.707308 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:18:30.707308 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:18:30.707308 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:18:30.707308 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:18:30.707308 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:18:30.707308 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:18:30.708391 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 19:18:30.708391 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 19:18:30.708391 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:18:30.708391 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:18:30.708391 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:18:30.708391 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:18:30.708391 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:18:30.708391 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:18:30.708391 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 620/1244] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 15 19:18:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:18:30.713286 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd4f837-0b2f-4f6e-94fa-83268c931c42 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:30.812734 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd4f837-0b2f-4f6e-94fa-83268c931c42 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:30.845395 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18740ced-cf9f-4383-a8d3-1389a85a3654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:30.846078 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18740ced-cf9f-4383-a8d3-1389a85a3654 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:30.846361 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18740ced-cf9f-4383-a8d3-1389a85a3654 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:30.846635 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18740ced-cf9f-4383-a8d3-1389a85a3654 None admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:30.847416 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18740ced-cf9f-4383-a8d3-1389a85a3654 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:18:30.847726 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18740ced-cf9f-4383-a8d3-1389a85a3654 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:30.849140 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18740ced-cf9f-4383-a8d3-1389a85a3654 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:30.861568 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 625/1245] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jul 15 19:18:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:18:30.867807 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce59a7a3-6a7d-45a3-88ff-a46564698c24 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:30.967684 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce59a7a3-6a7d-45a3-88ff-a46564698c24 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:31.000502 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06659e7-bb91-4dd1-908d-a0e8fd31c2f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:31.001148 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06659e7-bb91-4dd1-908d-a0e8fd31c2f8 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:31.001456 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06659e7-bb91-4dd1-908d-a0e8fd31c2f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:31.001798 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06659e7-bb91-4dd1-908d-a0e8fd31c2f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:31.002627 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06659e7-bb91-4dd1-908d-a0e8fd31c2f8 None admin] RBAC: Adding query filter params (role.id=70de044d6962417593b6d03571e7502e, scope.project.id=55a756ccaef14cb49b26b0d68dd0b466, user.id=52a0fce231ba4785a685ac9898d07503) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:18:31.002930 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06659e7-bb91-4dd1-908d-a0e8fd31c2f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:31.004346 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06659e7-bb91-4dd1-908d-a0e8fd31c2f8 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:31.010106 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 621/1246] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 15 19:18:30 2022] GET /identity/v3/role_assignments?user.id=52a0fce231ba4785a685ac9898d07503&scope.project.id=55a756ccaef14cb49b26b0d68dd0b466&role.id=70de044d6962417593b6d03571e7502e => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:18:33.068042 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02abf142-a3a6-4fa1-9d83-7a60f83773c7 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:33.068667 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02abf142-a3a6-4fa1-9d83-7a60f83773c7 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:33.068667 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02abf142-a3a6-4fa1-9d83-7a60f83773c7 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:33.069952 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 626/1247] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:18:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:18:33.075488 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df0521a-ebae-4bba-a3a4-af460642ed65 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:33.075998 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df0521a-ebae-4bba-a3a4-af460642ed65 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:33.076093 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df0521a-ebae-4bba-a3a4-af460642ed65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:33.081115 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:33.093600 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-4df0521a-ebae-4bba-a3a4-af460642ed65 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:18:33.094839 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-4df0521a-ebae-4bba-a3a4-af460642ed65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:18:33.160240 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 622/1248] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:18:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:18:33.183780 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02abf142-a3a6-4fa1-9d83-7a60f83773c7 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:33.202215 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:33.284199 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02abf142-a3a6-4fa1-9d83-7a60f83773c7 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:33.318548 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431f0821-fdce-4d7a-bdcf-f9fe8c9a1e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:33.319265 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431f0821-fdce-4d7a-bdcf-f9fe8c9a1e5c None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:33.319553 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431f0821-fdce-4d7a-bdcf-f9fe8c9a1e5c None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:33.319830 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431f0821-fdce-4d7a-bdcf-f9fe8c9a1e5c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:33.330474 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431f0821-fdce-4d7a-bdcf-f9fe8c9a1e5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:18:33.338056 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-431f0821-fdce-4d7a-bdcf-f9fe8c9a1e5c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:18:33.338056 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:18:33.338056 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:18:33.338056 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:18:33.338056 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:18:33.338056 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:18:33.338056 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:18:33.338056 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:18:33.338056 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:18:33.338056 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:18:33.338056 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:18:33.338056 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:18:33.338056 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:18:33.338056 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:18:33.338056 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:18:33.338056 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:18:33.339190 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:18:33.339190 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:18:33.339190 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:18:33.339190 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:18:33.339190 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:18:33.339190 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:18:33.339190 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:18:33.339190 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:18:33.339190 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:18:33.339190 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:18:33.339190 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:18:33.339190 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:18:33.339190 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:18:33.339190 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:18:33.339190 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:18:33.339190 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:18:33.340134 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:18:33.340134 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:18:33.340134 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:18:33.340134 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:18:33.340134 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:18:33.340134 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:18:33.340134 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:18:33.340134 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:18:33.340134 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:18:33.340134 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:18:33.340134 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:18:33.340134 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:18:33.340134 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 627/1249] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jul 15 19:18:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:18:33.345365 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df0521a-ebae-4bba-a3a4-af460642ed65 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:33.444121 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df0521a-ebae-4bba-a3a4-af460642ed65 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:33.473068 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:33.478010 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf35a8b5-47f1-4e2c-b655-52dc6d52e737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:33.478676 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf35a8b5-47f1-4e2c-b655-52dc6d52e737 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:33.478957 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf35a8b5-47f1-4e2c-b655-52dc6d52e737 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:33.479235 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf35a8b5-47f1-4e2c-b655-52dc6d52e737 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:33.480013 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf35a8b5-47f1-4e2c-b655-52dc6d52e737 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:18:33.480314 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf35a8b5-47f1-4e2c-b655-52dc6d52e737 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:33.481695 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf35a8b5-47f1-4e2c-b655-52dc6d52e737 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:33.489547 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 623/1250] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 15 19:18:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:18:33.496009 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431f0821-fdce-4d7a-bdcf-f9fe8c9a1e5c None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:33.572864 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:33.594983 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431f0821-fdce-4d7a-bdcf-f9fe8c9a1e5c None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:33.627766 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8523cb1a-b6bc-4fe3-aa1d-ff9bb87fe939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:33.628560 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8523cb1a-b6bc-4fe3-aa1d-ff9bb87fe939 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:33.628847 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8523cb1a-b6bc-4fe3-aa1d-ff9bb87fe939 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:33.629126 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8523cb1a-b6bc-4fe3-aa1d-ff9bb87fe939 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:33.640576 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8523cb1a-b6bc-4fe3-aa1d-ff9bb87fe939 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:33.642171 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8523cb1a-b6bc-4fe3-aa1d-ff9bb87fe939 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:33.662384 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-8523cb1a-b6bc-4fe3-aa1d-ff9bb87fe939 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:18:33.662384 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:18:33.662384 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:18:33.662384 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:18:33.662384 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:18:33.662384 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:18:33.662384 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:18:33.662384 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:18:33.662384 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:18:33.662384 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:18:33.662384 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:18:33.662384 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:18:33.662384 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:18:33.662384 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:18:33.662384 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:18:33.663433 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:18:33.663433 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:18:33.663433 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:18:33.663433 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:18:33.663433 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:18:33.663433 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:18:33.663433 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:18:33.663433 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:18:33.663433 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:18:33.663433 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:18:33.663433 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:18:33.663433 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:18:33.663433 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:18:33.663433 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:18:33.663433 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:18:33.663433 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:18:33.663433 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:18:33.664356 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:18:33.664356 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:18:33.664356 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:18:33.664356 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:18:33.664356 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 628/1251] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:18:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:18:33.670465 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf35a8b5-47f1-4e2c-b655-52dc6d52e737 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:33.769595 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf35a8b5-47f1-4e2c-b655-52dc6d52e737 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:33.803412 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c2c428-83bd-4a62-b5fa-27508247fa09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:33.813411 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c2c428-83bd-4a62-b5fa-27508247fa09 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:33.813411 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c2c428-83bd-4a62-b5fa-27508247fa09 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:33.813411 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c2c428-83bd-4a62-b5fa-27508247fa09 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:33.813411 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c2c428-83bd-4a62-b5fa-27508247fa09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:18:33.813411 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c2c428-83bd-4a62-b5fa-27508247fa09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:33.813411 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c2c428-83bd-4a62-b5fa-27508247fa09 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:33.819473 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 624/1252] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:18:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:18:33.825181 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8523cb1a-b6bc-4fe3-aa1d-ff9bb87fe939 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:33.924288 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8523cb1a-b6bc-4fe3-aa1d-ff9bb87fe939 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:33.949534 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:33.957356 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97aa6e4-6c0e-4eed-b557-bd8f59e4f5d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:33.958178 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97aa6e4-6c0e-4eed-b557-bd8f59e4f5d8 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:33.958460 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97aa6e4-6c0e-4eed-b557-bd8f59e4f5d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:33.959870 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97aa6e4-6c0e-4eed-b557-bd8f59e4f5d8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:33.966710 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97aa6e4-6c0e-4eed-b557-bd8f59e4f5d8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:33.968125 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97aa6e4-6c0e-4eed-b557-bd8f59e4f5d8 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:33.982523 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-a97aa6e4-6c0e-4eed-b557-bd8f59e4f5d8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 15 19:18:33.982523 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:18:33.982523 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:18:33.982523 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:18:33.982523 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:18:33.982523 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:18:33.982523 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:18:33.982523 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:18:33.982523 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:18:33.982523 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:18:33.982523 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:18:33.982523 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:18:33.982523 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 19:18:33.982523 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 19:18:33.982523 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 19:18:33.982523 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 19:18:33.983521 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:18:33.983521 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:18:33.983521 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:18:33.983521 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:18:33.983521 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:18:33.983521 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:18:33.983521 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:18:33.983521 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:18:33.983521 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:18:33.983521 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:18:33.983521 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:18:33.983521 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:18:33.983521 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:18:33.983521 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:18:33.983521 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:18:33.983521 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:18:33.983521 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:18:33.983521 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:18:33.984728 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:18:33.984728 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:18:33.984728 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 19:18:33.984728 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 19:18:33.984728 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 19:18:33.984728 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 19:18:33.984728 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 19:18:33.984728 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 19:18:33.984728 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 15 19:18:33.984728 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:18:33.984728 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 629/1253] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:18:33 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:18:33.989542 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c2c428-83bd-4a62-b5fa-27508247fa09 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:34.043787 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:34.086613 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c2c428-83bd-4a62-b5fa-27508247fa09 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:34.120045 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0934bde9-bd51-4f2b-a117-305083f391b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:34.120747 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0934bde9-bd51-4f2b-a117-305083f391b5 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:34.121027 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0934bde9-bd51-4f2b-a117-305083f391b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:34.121303 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0934bde9-bd51-4f2b-a117-305083f391b5 None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:34.122172 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0934bde9-bd51-4f2b-a117-305083f391b5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:18:34.122484 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0934bde9-bd51-4f2b-a117-305083f391b5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:34.123913 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0934bde9-bd51-4f2b-a117-305083f391b5 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:34.141060 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 625/1254] 10.222.0.33 () {56 vars in 1143 bytes} [Fri Jul 15 19:18:33 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:18:34.147016 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97aa6e4-6c0e-4eed-b557-bd8f59e4f5d8 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:34.245007 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97aa6e4-6c0e-4eed-b557-bd8f59e4f5d8 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:34.278701 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78321eae-d502-42ef-ba17-ed2b6cd27304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:34.280847 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78321eae-d502-42ef-ba17-ed2b6cd27304 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:34.280847 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78321eae-d502-42ef-ba17-ed2b6cd27304 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:34.280847 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78321eae-d502-42ef-ba17-ed2b6cd27304 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:34.334981 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:34.347513 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78321eae-d502-42ef-ba17-ed2b6cd27304 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:34.348891 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78321eae-d502-42ef-ba17-ed2b6cd27304 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:34.452366 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 630/1255] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 15 19:18:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 19:18:34.458305 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0934bde9-bd51-4f2b-a117-305083f391b5 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:34.557533 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0934bde9-bd51-4f2b-a117-305083f391b5 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:34.571142 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:34.591315 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1533068c-354b-4687-8034-9a0d82c8df29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:34.592096 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1533068c-354b-4687-8034-9a0d82c8df29 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:34.592373 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1533068c-354b-4687-8034-9a0d82c8df29 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:34.592642 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1533068c-354b-4687-8034-9a0d82c8df29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:34.603230 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1533068c-354b-4687-8034-9a0d82c8df29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:34.604096 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1533068c-354b-4687-8034-9a0d82c8df29 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:34.622131 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-1533068c-354b-4687-8034-9a0d82c8df29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:18:34.622131 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:18:34.622131 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:18:34.622131 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:18:34.622131 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:18:34.622131 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:18:34.622131 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:18:34.622131 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:18:34.622131 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:18:34.622131 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:18:34.622131 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:18:34.622131 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:18:34.622131 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:18:34.622131 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:18:34.622131 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:18:34.623191 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:18:34.623191 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:18:34.623191 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:18:34.623191 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:18:34.623191 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:18:34.623191 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:18:34.623191 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:18:34.623191 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:18:34.623191 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:18:34.623191 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:18:34.623191 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:18:34.623191 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:18:34.623191 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:18:34.623191 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:18:34.623191 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:18:34.623191 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:18:34.623191 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:18:34.624353 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:18:34.624353 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:18:34.624353 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:18:34.624353 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:18:34.624353 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 626/1256] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:18:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:18:34.628984 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78321eae-d502-42ef-ba17-ed2b6cd27304 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:34.716867 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:34.728302 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78321eae-d502-42ef-ba17-ed2b6cd27304 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:34.762566 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0782eeda-3e28-4c8d-9b93-26d46d87db98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:34.763227 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0782eeda-3e28-4c8d-9b93-26d46d87db98 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:34.763512 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0782eeda-3e28-4c8d-9b93-26d46d87db98 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:34.763791 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0782eeda-3e28-4c8d-9b93-26d46d87db98 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:34.764518 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0782eeda-3e28-4c8d-9b93-26d46d87db98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:18:34.765106 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0782eeda-3e28-4c8d-9b93-26d46d87db98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:34.766484 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0782eeda-3e28-4c8d-9b93-26d46d87db98 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:34.778810 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 631/1257] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:18:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:18:34.788393 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1533068c-354b-4687-8034-9a0d82c8df29 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:34.887509 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1533068c-354b-4687-8034-9a0d82c8df29 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:34.921822 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14cb543-e421-4539-96dc-1c32c5c0d635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:34.922602 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14cb543-e421-4539-96dc-1c32c5c0d635 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:34.922877 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14cb543-e421-4539-96dc-1c32c5c0d635 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:34.923146 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14cb543-e421-4539-96dc-1c32c5c0d635 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:34.929559 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14cb543-e421-4539-96dc-1c32c5c0d635 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:34.931013 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14cb543-e421-4539-96dc-1c32c5c0d635 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:34.944274 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-d14cb543-e421-4539-96dc-1c32c5c0d635 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:18:34.944274 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:18:34.944274 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:18:34.944274 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:18:34.944274 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:18:34.944274 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:18:34.944274 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:18:34.944274 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:18:34.944274 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:18:34.944274 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:18:34.944274 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:18:34.944274 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:18:34.944274 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 19:18:34.944274 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 19:18:34.944274 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 19:18:34.949974 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 19:18:34.949974 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:18:34.949974 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:18:34.949974 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:18:34.949974 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:18:34.949974 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:18:34.949974 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:18:34.949974 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:18:34.949974 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:18:34.949974 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:18:34.949974 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:18:34.949974 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:18:34.949974 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:18:34.949974 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:18:34.949974 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:18:34.949974 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:18:34.949974 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:18:34.950924 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 19:18:34.950924 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 19:18:34.950924 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:18:34.950924 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:18:34.950924 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:18:34.950924 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:18:34.950924 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:18:34.950924 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:18:34.950924 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 627/1258] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 15 19:18:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:18:34.951440 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0782eeda-3e28-4c8d-9b93-26d46d87db98 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:35.057718 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0782eeda-3e28-4c8d-9b93-26d46d87db98 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:35.094005 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede4ebc2-5f1b-4f00-a7aa-d450dbcebf7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:35.094669 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede4ebc2-5f1b-4f00-a7aa-d450dbcebf7a None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:35.094951 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede4ebc2-5f1b-4f00-a7aa-d450dbcebf7a None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:35.095226 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede4ebc2-5f1b-4f00-a7aa-d450dbcebf7a None admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:35.096013 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede4ebc2-5f1b-4f00-a7aa-d450dbcebf7a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:18:35.096320 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede4ebc2-5f1b-4f00-a7aa-d450dbcebf7a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:35.097747 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede4ebc2-5f1b-4f00-a7aa-d450dbcebf7a None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:35.110679 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 632/1259] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jul 15 19:18:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:18:35.116942 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14cb543-e421-4539-96dc-1c32c5c0d635 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:35.145240 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:35.219638 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14cb543-e421-4539-96dc-1c32c5c0d635 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:35.252136 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98e5e77-473b-4bb0-8fb9-9fbd20e27631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:35.252788 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e5e77-473b-4bb0-8fb9-9fbd20e27631 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:35.253070 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e5e77-473b-4bb0-8fb9-9fbd20e27631 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:35.253344 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e5e77-473b-4bb0-8fb9-9fbd20e27631 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:35.254244 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98e5e77-473b-4bb0-8fb9-9fbd20e27631 None admin] RBAC: Adding query filter params (role.id=4318f86ba9f74cd4a0bef1c311276657, scope.project.id=55a756ccaef14cb49b26b0d68dd0b466, user.id=52a0fce231ba4785a685ac9898d07503) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:18:35.254553 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98e5e77-473b-4bb0-8fb9-9fbd20e27631 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:35.255980 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98e5e77-473b-4bb0-8fb9-9fbd20e27631 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:35.261884 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 628/1260] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 15 19:18:35 2022] GET /identity/v3/role_assignments?user.id=52a0fce231ba4785a685ac9898d07503&scope.project.id=55a756ccaef14cb49b26b0d68dd0b466&role.id=4318f86ba9f74cd4a0bef1c311276657 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:18:37.244110 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c428cf5-ff62-4e9c-aa04-43d7c03f9a7b None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:37.244627 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c428cf5-ff62-4e9c-aa04-43d7c03f9a7b None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:37.244627 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c428cf5-ff62-4e9c-aa04-43d7c03f9a7b None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:37.246020 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 633/1261] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:18:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:18:37.251238 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849a597c-a600-4d88-9bac-9cada9569e91 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:37.251678 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849a597c-a600-4d88-9bac-9cada9569e91 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:37.251772 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849a597c-a600-4d88-9bac-9cada9569e91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:37.269127 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-849a597c-a600-4d88-9bac-9cada9569e91 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:18:37.270366 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-849a597c-a600-4d88-9bac-9cada9569e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:18:37.334794 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 629/1262] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:18:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:18:37.357591 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c428cf5-ff62-4e9c-aa04-43d7c03f9a7b None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:37.368409 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:37.463373 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c428cf5-ff62-4e9c-aa04-43d7c03f9a7b None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:37.501869 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acd3d78-8e55-4de8-9520-d8ff99558171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:37.502654 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acd3d78-8e55-4de8-9520-d8ff99558171 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:37.502934 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acd3d78-8e55-4de8-9520-d8ff99558171 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:37.503208 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acd3d78-8e55-4de8-9520-d8ff99558171 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:37.513621 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acd3d78-8e55-4de8-9520-d8ff99558171 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:18:37.521381 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-7acd3d78-8e55-4de8-9520-d8ff99558171 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:18:37.521381 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:18:37.521381 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:18:37.521381 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:18:37.521381 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:18:37.521381 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:18:37.521381 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:18:37.521381 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:18:37.521381 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:18:37.521381 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:18:37.521381 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:18:37.521381 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:18:37.521381 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:18:37.521381 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:18:37.521381 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:18:37.521381 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:18:37.522527 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:18:37.522527 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:18:37.522527 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:18:37.522527 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:18:37.522527 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:18:37.522527 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:18:37.522527 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:18:37.522527 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:18:37.522527 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:18:37.522527 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:18:37.522527 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:18:37.522527 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:18:37.522527 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:18:37.522527 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:18:37.522527 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:18:37.522527 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:18:37.523405 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:18:37.523405 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:18:37.523405 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:18:37.523405 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:18:37.523405 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:18:37.523405 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:18:37.523405 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:18:37.523405 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:18:37.523405 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:18:37.523405 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:18:37.523405 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:18:37.523405 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:18:37.523405 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 634/1263] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jul 15 19:18:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:18:37.529656 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849a597c-a600-4d88-9bac-9cada9569e91 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:37.547967 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:37.626027 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849a597c-a600-4d88-9bac-9cada9569e91 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:37.659785 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916ffba8-6607-41ad-96db-4c2ba58a46b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:37.660465 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916ffba8-6607-41ad-96db-4c2ba58a46b2 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:37.660750 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916ffba8-6607-41ad-96db-4c2ba58a46b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:37.661029 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916ffba8-6607-41ad-96db-4c2ba58a46b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:37.661860 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916ffba8-6607-41ad-96db-4c2ba58a46b2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:18:37.662169 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916ffba8-6607-41ad-96db-4c2ba58a46b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:37.663541 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916ffba8-6607-41ad-96db-4c2ba58a46b2 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:37.671641 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 630/1264] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 15 19:18:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:18:37.677526 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acd3d78-8e55-4de8-9520-d8ff99558171 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:37.749088 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:37.783981 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acd3d78-8e55-4de8-9520-d8ff99558171 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:37.817837 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1622d04-b766-4915-bc74-90847fd28243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:37.818647 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1622d04-b766-4915-bc74-90847fd28243 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:37.818926 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1622d04-b766-4915-bc74-90847fd28243 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:37.819200 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1622d04-b766-4915-bc74-90847fd28243 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:37.830134 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1622d04-b766-4915-bc74-90847fd28243 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:37.831569 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1622d04-b766-4915-bc74-90847fd28243 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:37.848354 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-b1622d04-b766-4915-bc74-90847fd28243 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:18:37.848354 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:18:37.848354 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:18:37.848354 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:18:37.848354 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:18:37.848354 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:18:37.848354 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:18:37.848354 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:18:37.848354 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:18:37.848354 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:18:37.848354 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:18:37.848354 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:18:37.848354 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:18:37.848354 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:18:37.848354 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:18:37.849410 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:18:37.849410 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:18:37.849410 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:18:37.849410 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:18:37.849410 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:18:37.849410 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:18:37.849410 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:18:37.849410 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:18:37.849410 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:18:37.849410 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:18:37.849410 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:18:37.849410 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:18:37.849410 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:18:37.849410 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:18:37.849410 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:18:37.849410 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:18:37.849410 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:18:37.850420 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:18:37.850420 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:18:37.850420 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:18:37.850420 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:18:37.850420 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 635/1265] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:18:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:18:37.855377 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916ffba8-6607-41ad-96db-4c2ba58a46b2 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:37.936674 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:37.955373 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916ffba8-6607-41ad-96db-4c2ba58a46b2 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:37.987002 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee032afc-c3c1-41bc-a52e-6dd17e086028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:37.987658 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee032afc-c3c1-41bc-a52e-6dd17e086028 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:37.987939 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee032afc-c3c1-41bc-a52e-6dd17e086028 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:37.988213 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee032afc-c3c1-41bc-a52e-6dd17e086028 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:37.988927 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee032afc-c3c1-41bc-a52e-6dd17e086028 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:18:37.989231 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee032afc-c3c1-41bc-a52e-6dd17e086028 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:37.990629 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee032afc-c3c1-41bc-a52e-6dd17e086028 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:38.002435 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 631/1266] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:18:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:18:38.008111 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1622d04-b766-4915-bc74-90847fd28243 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:38.107498 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1622d04-b766-4915-bc74-90847fd28243 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:38.126971 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:38.141086 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01381fd-5af6-4bc1-906b-0973c4e2731a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:38.142053 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01381fd-5af6-4bc1-906b-0973c4e2731a None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:38.142394 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01381fd-5af6-4bc1-906b-0973c4e2731a None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:38.142686 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01381fd-5af6-4bc1-906b-0973c4e2731a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:38.150292 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01381fd-5af6-4bc1-906b-0973c4e2731a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:38.151735 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01381fd-5af6-4bc1-906b-0973c4e2731a None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:38.165917 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-e01381fd-5af6-4bc1-906b-0973c4e2731a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 15 19:18:38.165917 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:18:38.165917 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:18:38.165917 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:18:38.165917 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:18:38.165917 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:18:38.165917 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:18:38.165917 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:18:38.165917 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:18:38.165917 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:18:38.165917 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:18:38.165917 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:18:38.165917 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 19:18:38.165917 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 19:18:38.165917 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 19:18:38.165917 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 19:18:38.166965 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:18:38.166965 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:18:38.166965 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:18:38.166965 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:18:38.166965 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:18:38.166965 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:18:38.166965 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:18:38.166965 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:18:38.166965 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:18:38.166965 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:18:38.166965 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:18:38.166965 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:18:38.166965 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:18:38.166965 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:18:38.166965 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:18:38.166965 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:18:38.166965 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:18:38.166965 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:18:38.168537 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:18:38.168537 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:18:38.168537 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 19:18:38.168537 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 19:18:38.168537 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 19:18:38.168537 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 19:18:38.168537 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 19:18:38.168537 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 19:18:38.168537 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 15 19:18:38.168537 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:18:38.168537 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 636/1267] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:18:38 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:18:38.173729 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee032afc-c3c1-41bc-a52e-6dd17e086028 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:38.272676 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee032afc-c3c1-41bc-a52e-6dd17e086028 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:38.306402 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4166b056-5321-4178-be25-0c39b9d3678f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:38.307074 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4166b056-5321-4178-be25-0c39b9d3678f None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:38.307351 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4166b056-5321-4178-be25-0c39b9d3678f None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:38.307626 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4166b056-5321-4178-be25-0c39b9d3678f None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:38.309026 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4166b056-5321-4178-be25-0c39b9d3678f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:18:38.309333 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4166b056-5321-4178-be25-0c39b9d3678f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:38.310798 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4166b056-5321-4178-be25-0c39b9d3678f None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:38.319344 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:38.328684 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 632/1268] 10.222.0.33 () {56 vars in 1143 bytes} [Fri Jul 15 19:18:38 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:18:38.334654 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01381fd-5af6-4bc1-906b-0973c4e2731a None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:38.433633 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01381fd-5af6-4bc1-906b-0973c4e2731a None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:38.466285 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7161a895-01ac-42c6-86fe-890a886e4eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:38.467070 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161a895-01ac-42c6-86fe-890a886e4eb0 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:38.467352 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161a895-01ac-42c6-86fe-890a886e4eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:38.467651 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161a895-01ac-42c6-86fe-890a886e4eb0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:38.479001 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7161a895-01ac-42c6-86fe-890a886e4eb0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:38.481038 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7161a895-01ac-42c6-86fe-890a886e4eb0 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:38.497883 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-7161a895-01ac-42c6-86fe-890a886e4eb0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:18:38.497883 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:18:38.497883 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:18:38.497883 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:18:38.497883 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:18:38.497883 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:18:38.497883 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:18:38.497883 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:18:38.497883 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:18:38.497883 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:18:38.497883 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:18:38.497883 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:18:38.497883 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:18:38.497883 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:18:38.497883 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:18:38.498950 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:18:38.498950 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:18:38.498950 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:18:38.498950 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:18:38.498950 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:18:38.498950 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:18:38.498950 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:18:38.498950 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:18:38.498950 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:18:38.498950 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:18:38.498950 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:18:38.498950 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:18:38.498950 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:18:38.498950 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:18:38.498950 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:18:38.498950 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:18:38.498950 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:18:38.500089 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:18:38.500089 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:18:38.500089 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:18:38.500089 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:18:38.500089 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 637/1269] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:18:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:18:38.505046 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4166b056-5321-4178-be25-0c39b9d3678f None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:38.602799 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4166b056-5321-4178-be25-0c39b9d3678f None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:38.636396 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d05883-beda-462a-839a-f940e2bea00c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:38.637070 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d05883-beda-462a-839a-f940e2bea00c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:38.637362 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d05883-beda-462a-839a-f940e2bea00c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:38.637636 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d05883-beda-462a-839a-f940e2bea00c None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:38.638379 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d05883-beda-462a-839a-f940e2bea00c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:18:38.638681 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d05883-beda-462a-839a-f940e2bea00c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:38.640309 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d05883-beda-462a-839a-f940e2bea00c None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:38.652103 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 633/1270] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:18:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:18:38.657784 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7161a895-01ac-42c6-86fe-890a886e4eb0 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:38.689633 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:38.755791 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7161a895-01ac-42c6-86fe-890a886e4eb0 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:38.790975 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e15b798-61bf-41fd-9c11-cff307cf3e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:38.791770 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e15b798-61bf-41fd-9c11-cff307cf3e9b None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:38.792065 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e15b798-61bf-41fd-9c11-cff307cf3e9b None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:38.792348 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e15b798-61bf-41fd-9c11-cff307cf3e9b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:38.798992 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e15b798-61bf-41fd-9c11-cff307cf3e9b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:38.800419 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e15b798-61bf-41fd-9c11-cff307cf3e9b None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:38.815289 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-4e15b798-61bf-41fd-9c11-cff307cf3e9b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:18:38.815289 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:18:38.815289 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:18:38.815289 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:18:38.815289 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:18:38.815289 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:18:38.815289 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:18:38.815289 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:18:38.815289 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:18:38.815289 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:18:38.815289 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:18:38.815289 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:18:38.815289 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 19:18:38.815289 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 19:18:38.815289 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 19:18:38.816346 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 19:18:38.816346 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:18:38.816346 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:18:38.816346 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:18:38.816346 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:18:38.816346 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:18:38.816346 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:18:38.816346 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:18:38.816346 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:18:38.816346 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:18:38.816346 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:18:38.816346 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:18:38.816346 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:18:38.816346 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:18:38.816346 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:18:38.816346 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:18:38.816346 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:18:38.817381 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 19:18:38.817381 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 19:18:38.817381 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:18:38.817381 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:18:38.817381 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:18:38.817381 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:18:38.817381 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:18:38.817381 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:18:38.817381 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 638/1271] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 15 19:18:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:18:38.822338 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d05883-beda-462a-839a-f940e2bea00c None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:38.882471 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:38.923009 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d05883-beda-462a-839a-f940e2bea00c None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:38.955465 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5d89a4-11e3-4ad4-b91c-d09832d4c75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:38.956038 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5d89a4-11e3-4ad4-b91c-d09832d4c75d None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:38.956320 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5d89a4-11e3-4ad4-b91c-d09832d4c75d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:38.956587 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5d89a4-11e3-4ad4-b91c-d09832d4c75d None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:38.957362 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5d89a4-11e3-4ad4-b91c-d09832d4c75d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:18:38.957664 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5d89a4-11e3-4ad4-b91c-d09832d4c75d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:38.959116 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5d89a4-11e3-4ad4-b91c-d09832d4c75d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:38.971125 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 634/1272] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jul 15 19:18:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:18:38.977293 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e15b798-61bf-41fd-9c11-cff307cf3e9b None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:39.065348 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:39.074137 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e15b798-61bf-41fd-9c11-cff307cf3e9b None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:39.107131 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a03bc6e-24f9-40df-9f93-cd3f5084d965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:39.108040 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a03bc6e-24f9-40df-9f93-cd3f5084d965 None admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:39.108321 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a03bc6e-24f9-40df-9f93-cd3f5084d965 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:39.108598 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a03bc6e-24f9-40df-9f93-cd3f5084d965 None admin] PATH_INFO: `/v3/projects/55a756ccaef14cb49b26b0d68dd0b466/users/52a0fce231ba4785a685ac9898d07503/roles/4318f86ba9f74cd4a0bef1c311276657` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:39.110563 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a03bc6e-24f9-40df-9f93-cd3f5084d965 None admin] RBAC: Authorizing `identity:create_grant(project_id=55a756ccaef14cb49b26b0d68dd0b466, user_id=52a0fce231ba4785a685ac9898d07503, role_id=4318f86ba9f74cd4a0bef1c311276657)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:39.111977 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a03bc6e-24f9-40df-9f93-cd3f5084d965 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:39.196667 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 639/1273] 10.222.0.33 () {58 vars in 1439 bytes} [Fri Jul 15 19:18:38 2022] PUT /identity/v3/projects/55a756ccaef14cb49b26b0d68dd0b466/users/52a0fce231ba4785a685ac9898d07503/roles/4318f86ba9f74cd4a0bef1c311276657 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:18:41.195296 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093254fa-b451-4bcf-8993-cc09c99e52f7 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:41.195807 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093254fa-b451-4bcf-8993-cc09c99e52f7 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:41.195807 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093254fa-b451-4bcf-8993-cc09c99e52f7 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:41.197155 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 635/1274] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:18:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:18:41.202330 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22975d19-bbcd-47b1-9793-06b4412c3ba3 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:41.202789 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22975d19-bbcd-47b1-9793-06b4412c3ba3 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:41.202884 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22975d19-bbcd-47b1-9793-06b4412c3ba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:41.219361 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-22975d19-bbcd-47b1-9793-06b4412c3ba3 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:18:41.220527 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-22975d19-bbcd-47b1-9793-06b4412c3ba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:18:41.288023 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 640/1275] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:18:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:18:41.312350 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093254fa-b451-4bcf-8993-cc09c99e52f7 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:41.414065 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093254fa-b451-4bcf-8993-cc09c99e52f7 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:41.431616 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:41.448095 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4066d3ab-beb6-4387-a928-5941df365305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:41.448819 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066d3ab-beb6-4387-a928-5941df365305 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:41.449098 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066d3ab-beb6-4387-a928-5941df365305 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:41.449370 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066d3ab-beb6-4387-a928-5941df365305 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:41.459465 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4066d3ab-beb6-4387-a928-5941df365305 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:18:41.467530 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-4066d3ab-beb6-4387-a928-5941df365305 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:18:41.467530 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:18:41.467530 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:18:41.467530 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:18:41.467530 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:18:41.467530 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:18:41.467530 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:18:41.467530 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:18:41.467530 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:18:41.467530 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:18:41.467530 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:18:41.467530 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:18:41.467530 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:18:41.467530 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:18:41.467530 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:18:41.467530 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:18:41.471108 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:18:41.471108 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:18:41.471108 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:18:41.471108 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:18:41.471108 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:18:41.471108 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:18:41.471108 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:18:41.471108 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:18:41.471108 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:18:41.471108 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:18:41.471108 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:18:41.471108 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:18:41.471108 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:18:41.471108 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:18:41.471108 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:18:41.471108 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:18:41.472075 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:18:41.472075 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:18:41.472075 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:18:41.472075 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:18:41.472075 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:18:41.472075 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:18:41.472075 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:18:41.472075 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:18:41.472075 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:18:41.472075 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:18:41.472075 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:18:41.472075 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:18:41.472075 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 636/1276] 10.222.0.33 () {56 vars in 1096 bytes} [Fri Jul 15 19:18:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:18:41.474970 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22975d19-bbcd-47b1-9793-06b4412c3ba3 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:41.554488 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:41.571794 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22975d19-bbcd-47b1-9793-06b4412c3ba3 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:41.605035 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb14cf0e-0e5b-4264-a090-d0e119e8eb5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:41.605690 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb14cf0e-0e5b-4264-a090-d0e119e8eb5c None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:41.605999 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb14cf0e-0e5b-4264-a090-d0e119e8eb5c None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:41.606272 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb14cf0e-0e5b-4264-a090-d0e119e8eb5c None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:41.607009 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb14cf0e-0e5b-4264-a090-d0e119e8eb5c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:18:41.607314 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb14cf0e-0e5b-4264-a090-d0e119e8eb5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:41.608674 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb14cf0e-0e5b-4264-a090-d0e119e8eb5c None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:41.616472 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 641/1277] 10.222.0.33 () {56 vars in 1099 bytes} [Fri Jul 15 19:18:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:18:41.622375 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4066d3ab-beb6-4387-a928-5941df365305 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:41.721131 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4066d3ab-beb6-4387-a928-5941df365305 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:41.758411 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e8d778-f83c-4195-aa54-82615de2232f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:41.759774 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8d778-f83c-4195-aa54-82615de2232f None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:41.760284 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8d778-f83c-4195-aa54-82615de2232f None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:41.761279 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8d778-f83c-4195-aa54-82615de2232f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:41.775946 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e8d778-f83c-4195-aa54-82615de2232f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:41.778041 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e8d778-f83c-4195-aa54-82615de2232f None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:41.795348 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-17e8d778-f83c-4195-aa54-82615de2232f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:18:41.795348 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:18:41.795348 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:18:41.795348 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:18:41.795348 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:18:41.795348 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:18:41.795348 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:18:41.795348 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:18:41.795348 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:18:41.795348 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:18:41.795348 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:18:41.795348 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:18:41.795348 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:18:41.795348 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:18:41.795348 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:18:41.796416 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:18:41.796416 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:18:41.796416 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:18:41.796416 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:18:41.796416 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:18:41.796416 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:18:41.796416 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:18:41.796416 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:18:41.796416 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:18:41.796416 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:18:41.796416 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:18:41.796416 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:18:41.796416 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:18:41.796416 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:18:41.796416 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:18:41.796416 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:18:41.796416 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:18:41.797479 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:18:41.797479 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:18:41.797479 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:18:41.797479 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:18:41.797479 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 637/1278] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:18:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:18:41.802505 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb14cf0e-0e5b-4264-a090-d0e119e8eb5c None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:41.902428 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb14cf0e-0e5b-4264-a090-d0e119e8eb5c None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:41.934147 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c289069b-e84c-4187-9b69-f81c09de0ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:41.934795 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c289069b-e84c-4187-9b69-f81c09de0ead None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:41.935075 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c289069b-e84c-4187-9b69-f81c09de0ead None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:41.935349 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c289069b-e84c-4187-9b69-f81c09de0ead None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:41.936061 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c289069b-e84c-4187-9b69-f81c09de0ead None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:18:41.936361 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c289069b-e84c-4187-9b69-f81c09de0ead None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:41.937703 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c289069b-e84c-4187-9b69-f81c09de0ead None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:41.950319 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 642/1279] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:18:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:18:41.956049 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e8d778-f83c-4195-aa54-82615de2232f None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:41.988435 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:42.058075 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e8d778-f83c-4195-aa54-82615de2232f None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:42.091721 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ffa8b8-e7e2-455e-b048-c265bad836c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:42.099873 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ffa8b8-e7e2-455e-b048-c265bad836c0 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:42.099873 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ffa8b8-e7e2-455e-b048-c265bad836c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:42.099873 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ffa8b8-e7e2-455e-b048-c265bad836c0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:42.100491 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ffa8b8-e7e2-455e-b048-c265bad836c0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:42.101961 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ffa8b8-e7e2-455e-b048-c265bad836c0 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:42.118857 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-67ffa8b8-e7e2-455e-b048-c265bad836c0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 15 19:18:42.118857 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:18:42.118857 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:18:42.118857 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:18:42.118857 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:18:42.118857 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:18:42.118857 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:18:42.118857 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:18:42.118857 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:18:42.118857 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:18:42.118857 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:18:42.118857 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:18:42.118857 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 19:18:42.118857 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 19:18:42.118857 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 19:18:42.118857 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 19:18:42.120566 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:18:42.120566 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:18:42.120566 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:18:42.120566 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:18:42.120566 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:18:42.120566 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:18:42.120566 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:18:42.120566 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:18:42.120566 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:18:42.120566 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:18:42.120566 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:18:42.120566 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:18:42.120566 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:18:42.120566 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:18:42.120566 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:18:42.120566 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:18:42.120566 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:18:42.120566 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:18:42.121750 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:18:42.121750 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:18:42.121750 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 19:18:42.121750 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 19:18:42.121750 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 19:18:42.121750 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 19:18:42.121750 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 19:18:42.121750 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 19:18:42.121750 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 15 19:18:42.121750 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:18:42.121750 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 638/1280] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:18:41 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:18:42.127034 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c289069b-e84c-4187-9b69-f81c09de0ead None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:42.131271 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:42.225121 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c289069b-e84c-4187-9b69-f81c09de0ead None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:42.258297 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ca0ba8-e681-4883-ad2f-377819933002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:42.258898 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca0ba8-e681-4883-ad2f-377819933002 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:42.259173 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca0ba8-e681-4883-ad2f-377819933002 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:42.259438 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca0ba8-e681-4883-ad2f-377819933002 None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:42.260263 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ca0ba8-e681-4883-ad2f-377819933002 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:18:42.260567 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ca0ba8-e681-4883-ad2f-377819933002 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:42.263526 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ca0ba8-e681-4883-ad2f-377819933002 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:42.280434 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 643/1281] 10.222.0.33 () {56 vars in 1143 bytes} [Fri Jul 15 19:18:42 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:18:42.286280 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ffa8b8-e7e2-455e-b048-c265bad836c0 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:42.376642 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:42.386382 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ffa8b8-e7e2-455e-b048-c265bad836c0 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:42.418493 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af2fb28-1b48-4364-9f01-48200128faa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:42.419138 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af2fb28-1b48-4364-9f01-48200128faa5 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:42.419418 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af2fb28-1b48-4364-9f01-48200128faa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:42.419689 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af2fb28-1b48-4364-9f01-48200128faa5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:42.485257 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af2fb28-1b48-4364-9f01-48200128faa5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:42.487763 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af2fb28-1b48-4364-9f01-48200128faa5 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:42.591301 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 639/1282] 10.222.0.33 () {58 vars in 1282 bytes} [Fri Jul 15 19:18:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 19:18:42.597251 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ca0ba8-e681-4883-ad2f-377819933002 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:42.658129 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:42.693931 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ca0ba8-e681-4883-ad2f-377819933002 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:42.726751 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d336cff0-f642-4db0-ba52-e8285b26468b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:42.727534 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d336cff0-f642-4db0-ba52-e8285b26468b None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:42.727818 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d336cff0-f642-4db0-ba52-e8285b26468b None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:42.728098 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d336cff0-f642-4db0-ba52-e8285b26468b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:42.737777 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d336cff0-f642-4db0-ba52-e8285b26468b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:42.739216 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d336cff0-f642-4db0-ba52-e8285b26468b None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:42.756148 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-d336cff0-f642-4db0-ba52-e8285b26468b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:18:42.756148 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:18:42.756148 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:18:42.756148 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:18:42.756148 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:18:42.756148 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:18:42.756148 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:18:42.756148 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:18:42.756148 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:18:42.756148 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:18:42.756148 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:18:42.756148 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:18:42.756148 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:18:42.756148 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:18:42.756148 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:18:42.757206 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:18:42.757206 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:18:42.757206 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:18:42.757206 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:18:42.757206 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:18:42.757206 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:18:42.757206 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:18:42.757206 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:18:42.757206 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:18:42.757206 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:18:42.757206 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:18:42.757206 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:18:42.757206 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:18:42.757206 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:18:42.757206 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:18:42.757206 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:18:42.757206 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:18:42.758414 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:18:42.758414 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:18:42.758414 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:18:42.758414 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:18:42.758414 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 644/1283] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:18:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:18:42.763543 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af2fb28-1b48-4364-9f01-48200128faa5 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:42.862301 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af2fb28-1b48-4364-9f01-48200128faa5 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:42.895928 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1a5c2f-6f6b-4190-9ab4-2a307fbaaebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:42.896594 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1a5c2f-6f6b-4190-9ab4-2a307fbaaebd None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:42.896879 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1a5c2f-6f6b-4190-9ab4-2a307fbaaebd None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:42.897157 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1a5c2f-6f6b-4190-9ab4-2a307fbaaebd None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:42.897907 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1a5c2f-6f6b-4190-9ab4-2a307fbaaebd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:18:42.898217 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1a5c2f-6f6b-4190-9ab4-2a307fbaaebd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:42.899584 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1a5c2f-6f6b-4190-9ab4-2a307fbaaebd None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:42.910444 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 640/1284] 10.222.0.33 () {56 vars in 1109 bytes} [Fri Jul 15 19:18:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:18:42.916193 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d336cff0-f642-4db0-ba52-e8285b26468b None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:43.013471 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d336cff0-f642-4db0-ba52-e8285b26468b None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:43.037200 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:43.046107 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350e93c6-aafa-49c5-9de0-142e94f850da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:43.047006 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350e93c6-aafa-49c5-9de0-142e94f850da None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:43.047295 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350e93c6-aafa-49c5-9de0-142e94f850da None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:43.047575 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350e93c6-aafa-49c5-9de0-142e94f850da None admin] PATH_INFO: `/v3/projects/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:43.054166 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350e93c6-aafa-49c5-9de0-142e94f850da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:43.055607 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350e93c6-aafa-49c5-9de0-142e94f850da None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:43.069483 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-350e93c6-aafa-49c5-9de0-142e94f850da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:18:43.069483 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:18:43.069483 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:18:43.069483 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:18:43.069483 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:18:43.069483 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:18:43.069483 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:18:43.069483 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:18:43.069483 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:18:43.069483 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:18:43.069483 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:18:43.069483 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:18:43.069483 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 19:18:43.069483 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 19:18:43.069483 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 19:18:43.074099 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 19:18:43.074099 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:18:43.074099 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:18:43.074099 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:18:43.074099 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:18:43.074099 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:18:43.074099 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:18:43.074099 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:18:43.074099 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Jul 15 19:18:43.074099 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:18:43.074099 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:18:43.074099 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:18:43.074099 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:18:43.074099 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:18:43.074099 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:18:43.074099 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:18:43.074099 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:18:43.075232 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 19:18:43.075232 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 19:18:43.075232 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:18:43.075232 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:18:43.075232 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:18:43.075232 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:18:43.075232 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:18:43.075232 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Jul 15 19:18:43.075232 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 645/1285] 10.222.0.33 () {56 vars in 1111 bytes} [Fri Jul 15 19:18:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:18:43.076805 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1a5c2f-6f6b-4190-9ab4-2a307fbaaebd None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:43.123430 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:43.177099 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1a5c2f-6f6b-4190-9ab4-2a307fbaaebd None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:43.192215 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:43.211400 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bfa84c-afe3-495a-a0c7-65a043bfe0fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:43.212052 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bfa84c-afe3-495a-a0c7-65a043bfe0fd None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:43.212331 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bfa84c-afe3-495a-a0c7-65a043bfe0fd None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:43.212604 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bfa84c-afe3-495a-a0c7-65a043bfe0fd None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:43.213376 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bfa84c-afe3-495a-a0c7-65a043bfe0fd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:18:43.214193 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bfa84c-afe3-495a-a0c7-65a043bfe0fd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:43.215623 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bfa84c-afe3-495a-a0c7-65a043bfe0fd None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:43.228056 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 641/1286] 10.222.0.33 () {56 vars in 1148 bytes} [Fri Jul 15 19:18:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:18:43.234862 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350e93c6-aafa-49c5-9de0-142e94f850da None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:43.335757 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350e93c6-aafa-49c5-9de0-142e94f850da None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:43.368752 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f52445-0d93-4d9f-a975-0d773fe01900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:43.369389 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f52445-0d93-4d9f-a975-0d773fe01900 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:43.369667 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f52445-0d93-4d9f-a975-0d773fe01900 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:43.369994 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f52445-0d93-4d9f-a975-0d773fe01900 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:43.370816 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f52445-0d93-4d9f-a975-0d773fe01900 None admin] RBAC: Adding query filter params (role.id=4318f86ba9f74cd4a0bef1c311276657, scope.project.id=55a756ccaef14cb49b26b0d68dd0b466, user.id=52a0fce231ba4785a685ac9898d07503) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:18:43.371119 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f52445-0d93-4d9f-a975-0d773fe01900 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:43.372521 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f52445-0d93-4d9f-a975-0d773fe01900 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:43.378375 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 646/1287] 10.222.0.33 () {56 vars in 1374 bytes} [Fri Jul 15 19:18:43 2022] GET /identity/v3/role_assignments?user.id=52a0fce231ba4785a685ac9898d07503&scope.project.id=55a756ccaef14cb49b26b0d68dd0b466&role.id=4318f86ba9f74cd4a0bef1c311276657 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:18:45.398625 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df35e8b7-35f1-4794-9fc4-6e60300fd5df None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:45.399186 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df35e8b7-35f1-4794-9fc4-6e60300fd5df None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:45.399186 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df35e8b7-35f1-4794-9fc4-6e60300fd5df None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:45.400543 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 642/1288] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:18:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:18:45.405671 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e32e862-8efa-49ad-897b-a8d4864e19f1 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:45.406159 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e32e862-8efa-49ad-897b-a8d4864e19f1 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:45.406265 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e32e862-8efa-49ad-897b-a8d4864e19f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:45.423645 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-1e32e862-8efa-49ad-897b-a8d4864e19f1 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:18:45.425158 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-1e32e862-8efa-49ad-897b-a8d4864e19f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:18:45.455516 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:45.491973 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 647/1289] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:18:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:18:45.515112 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df35e8b7-35f1-4794-9fc4-6e60300fd5df None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:45.617227 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df35e8b7-35f1-4794-9fc4-6e60300fd5df None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:45.628353 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:45.650993 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783a3b11-6f47-4644-acb1-aa976c9f0804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:45.651722 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783a3b11-6f47-4644-acb1-aa976c9f0804 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:45.652004 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783a3b11-6f47-4644-acb1-aa976c9f0804 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:45.652278 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783a3b11-6f47-4644-acb1-aa976c9f0804 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:45.652926 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783a3b11-6f47-4644-acb1-aa976c9f0804 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:45.654328 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783a3b11-6f47-4644-acb1-aa976c9f0804 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:45.665835 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-783a3b11-6f47-4644-acb1-aa976c9f0804 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 15 19:18:45.665835 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:18:45.665835 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 19:18:45.665835 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 19:18:45.665835 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 19:18:45.665835 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 19:18:45.665835 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 19:18:45.665835 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:18:45.665835 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 15 19:18:45.665835 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:18:45.665835 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 19:18:45.665835 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:18:45.665835 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:18:45.665835 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:18:45.665835 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:18:45.665835 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:18:45.665835 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:18:45.667618 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:18:45.667618 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:18:45.667618 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:18:45.667618 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:18:45.667618 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:18:45.667618 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:18:45.667618 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 19:18:45.667618 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 19:18:45.667618 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 19:18:45.667618 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 19:18:45.667618 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:18:45.667618 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:18:45.667618 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:18:45.667618 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:18:45.667618 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:18:45.667618 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:18:45.668588 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:18:45.668588 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:18:45.668588 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:18:45.668588 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:18:45.668588 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:18:45.668588 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:18:45.668588 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:18:45.668588 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:18:45.668588 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:18:45.668588 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:18:45.668588 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 19:18:45.668588 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:18:45.668588 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 15 19:18:45.668588 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:18:45.668588 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 643/1290] 10.222.0.33 () {56 vars in 1117 bytes} [Fri Jul 15 19:18:45 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:18:45.673814 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e32e862-8efa-49ad-897b-a8d4864e19f1 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:45.770995 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e32e862-8efa-49ad-897b-a8d4864e19f1 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:45.810318 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3540e1-e7ab-4ef8-b8f5-7d442852b8b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:45.810993 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3540e1-e7ab-4ef8-b8f5-7d442852b8b5 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:45.811274 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3540e1-e7ab-4ef8-b8f5-7d442852b8b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:45.811548 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3540e1-e7ab-4ef8-b8f5-7d442852b8b5 None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:45.812341 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3540e1-e7ab-4ef8-b8f5-7d442852b8b5 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:18:45.812565 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3540e1-e7ab-4ef8-b8f5-7d442852b8b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:45.813982 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3540e1-e7ab-4ef8-b8f5-7d442852b8b5 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:45.818948 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 648/1291] 10.222.0.33 () {56 vars in 1116 bytes} [Fri Jul 15 19:18:45 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:18:45.824912 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783a3b11-6f47-4644-acb1-aa976c9f0804 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:45.926362 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783a3b11-6f47-4644-acb1-aa976c9f0804 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:45.958619 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054eb7cb-1eb7-4495-895c-0b9c404ce43a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:45.959282 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054eb7cb-1eb7-4495-895c-0b9c404ce43a None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:45.959563 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054eb7cb-1eb7-4495-895c-0b9c404ce43a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:45.959839 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054eb7cb-1eb7-4495-895c-0b9c404ce43a None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:45.960544 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054eb7cb-1eb7-4495-895c-0b9c404ce43a None admin] RBAC: Adding query filter params (type=placement) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:18:45.960851 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054eb7cb-1eb7-4495-895c-0b9c404ce43a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:45.962264 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054eb7cb-1eb7-4495-895c-0b9c404ce43a None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:45.967213 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 644/1292] 10.222.0.33 () {56 vars in 1116 bytes} [Fri Jul 15 19:18:45 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:18:47.939835 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97904b1b-1786-453a-83a7-b52d003d7c6d None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:47.940374 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97904b1b-1786-453a-83a7-b52d003d7c6d None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:47.940374 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97904b1b-1786-453a-83a7-b52d003d7c6d None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:47.941805 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 649/1293] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:18:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:18:47.947028 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace64d26-8a16-422d-b169-883aaabdf6fc None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:47.947543 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace64d26-8a16-422d-b169-883aaabdf6fc None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:47.947641 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace64d26-8a16-422d-b169-883aaabdf6fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:47.965008 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-ace64d26-8a16-422d-b169-883aaabdf6fc None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:18:47.966222 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-ace64d26-8a16-422d-b169-883aaabdf6fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:18:47.994671 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:48.031137 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 645/1294] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:18:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:18:48.053035 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97904b1b-1786-453a-83a7-b52d003d7c6d None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:48.087465 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:48.157305 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97904b1b-1786-453a-83a7-b52d003d7c6d None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:48.189886 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d518294-f8c3-4fb4-af05-69d647a5748b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:48.190544 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d518294-f8c3-4fb4-af05-69d647a5748b None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:48.190823 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d518294-f8c3-4fb4-af05-69d647a5748b None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:48.191094 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d518294-f8c3-4fb4-af05-69d647a5748b None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:48.191813 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d518294-f8c3-4fb4-af05-69d647a5748b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:48.193158 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d518294-f8c3-4fb4-af05-69d647a5748b None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:48.345698 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 650/1295] 10.222.0.33 () {60 vars in 1141 bytes} [Fri Jul 15 19:18:48 2022] POST /identity/v3/services => generated 246 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:18:50.390251 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e29334-9925-4418-b1f7-03e71ec5a6e5 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:50.390840 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e29334-9925-4418-b1f7-03e71ec5a6e5 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:50.390840 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e29334-9925-4418-b1f7-03e71ec5a6e5 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:50.392185 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 646/1296] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:18:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:18:50.397391 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3327a0-f591-4dc1-979a-e20be9c5061f None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:50.397794 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3327a0-f591-4dc1-979a-e20be9c5061f None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:50.397991 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3327a0-f591-4dc1-979a-e20be9c5061f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:50.415453 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-9b3327a0-f591-4dc1-979a-e20be9c5061f None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:18:50.416839 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-9b3327a0-f591-4dc1-979a-e20be9c5061f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:18:50.489676 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 651/1297] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:18:50 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 19:18:50.512192 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e29334-9925-4418-b1f7-03e71ec5a6e5 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:50.616556 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e29334-9925-4418-b1f7-03e71ec5a6e5 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:50.650462 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f835a3-6f73-42dd-96ba-f030fb611330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:50.651689 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f835a3-6f73-42dd-96ba-f030fb611330 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:50.651964 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f835a3-6f73-42dd-96ba-f030fb611330 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:50.652232 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f835a3-6f73-42dd-96ba-f030fb611330 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:50.652871 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f835a3-6f73-42dd-96ba-f030fb611330 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:50.654276 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f835a3-6f73-42dd-96ba-f030fb611330 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:50.665991 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-d1f835a3-6f73-42dd-96ba-f030fb611330 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 15 19:18:50.665991 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:18:50.665991 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 19:18:50.665991 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 19:18:50.665991 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 19:18:50.665991 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 19:18:50.665991 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 19:18:50.665991 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:18:50.665991 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 15 19:18:50.665991 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:18:50.665991 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 19:18:50.665991 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:18:50.665991 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:18:50.665991 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:18:50.665991 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:18:50.665991 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:18:50.665991 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:18:50.670824 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:18:50.670824 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:18:50.670824 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:18:50.670824 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:18:50.670824 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:18:50.670824 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:18:50.670824 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 19:18:50.670824 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 19:18:50.670824 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 19:18:50.670824 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 19:18:50.670824 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:18:50.670824 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:18:50.670824 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:18:50.670824 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:18:50.670824 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:18:50.670824 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:18:50.671710 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:18:50.671710 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:18:50.671710 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:18:50.671710 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:18:50.671710 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:18:50.671710 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:18:50.671710 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:18:50.671710 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:18:50.671710 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:18:50.671710 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:18:50.671710 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 19:18:50.671710 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:18:50.671710 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 15 19:18:50.671710 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:18:50.671710 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 647/1298] 10.222.0.33 () {56 vars in 1117 bytes} [Fri Jul 15 19:18:50 2022] GET /identity/v3/services/placement => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:18:50.674153 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3327a0-f591-4dc1-979a-e20be9c5061f None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:50.692439 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:50.771387 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3327a0-f591-4dc1-979a-e20be9c5061f None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:50.804917 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff992eb-5050-4c15-99f8-122ee889b82c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:50.805598 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff992eb-5050-4c15-99f8-122ee889b82c None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:50.805903 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff992eb-5050-4c15-99f8-122ee889b82c None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:50.806176 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff992eb-5050-4c15-99f8-122ee889b82c None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:50.807142 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff992eb-5050-4c15-99f8-122ee889b82c None admin] RBAC: Adding query filter params (name=placement) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:18:50.807142 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff992eb-5050-4c15-99f8-122ee889b82c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:50.808489 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff992eb-5050-4c15-99f8-122ee889b82c None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:50.813803 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 652/1299] 10.222.0.33 () {56 vars in 1116 bytes} [Fri Jul 15 19:18:50 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:18:50.819804 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f835a3-6f73-42dd-96ba-f030fb611330 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:50.851344 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:50.919510 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f835a3-6f73-42dd-96ba-f030fb611330 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:50.952166 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faa5c52-66ec-4ea7-9db3-24c507f71f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:50.952818 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa5c52-66ec-4ea7-9db3-24c507f71f37 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:50.953101 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa5c52-66ec-4ea7-9db3-24c507f71f37 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:50.953378 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa5c52-66ec-4ea7-9db3-24c507f71f37 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:50.954196 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faa5c52-66ec-4ea7-9db3-24c507f71f37 None admin] RBAC: Adding query filter params (interface=public, service_id=015ad36bcbe34891b845f890a10ece4f, region_id=RegionOne) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:18:50.954510 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faa5c52-66ec-4ea7-9db3-24c507f71f37 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:50.955877 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faa5c52-66ec-4ea7-9db3-24c507f71f37 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:50.960819 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 648/1300] 10.222.0.33 () {56 vars in 1251 bytes} [Fri Jul 15 19:18:50 2022] GET /identity/v3/endpoints?service_id=015ad36bcbe34891b845f890a10ece4f&interface=public®ion_id=RegionOne => generated 194 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:18:50.966640 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff992eb-5050-4c15-99f8-122ee889b82c None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:51.048047 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:51.066253 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff992eb-5050-4c15-99f8-122ee889b82c None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:51.098799 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092757c5-89fb-4ea2-8746-d38d0935cbea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:51.099459 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092757c5-89fb-4ea2-8746-d38d0935cbea None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:51.099743 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092757c5-89fb-4ea2-8746-d38d0935cbea None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:51.100019 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092757c5-89fb-4ea2-8746-d38d0935cbea None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:51.100702 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092757c5-89fb-4ea2-8746-d38d0935cbea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:51.102089 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092757c5-89fb-4ea2-8746-d38d0935cbea None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:51.109407 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 653/1301] 10.222.0.33 () {56 vars in 1087 bytes} [Fri Jul 15 19:18:50 2022] GET /identity/v3/services => generated 2191 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:18:53.136303 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53fd43c-5cf3-4458-a2a2-1a94c4ac9427 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:53.136803 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53fd43c-5cf3-4458-a2a2-1a94c4ac9427 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:53.136803 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53fd43c-5cf3-4458-a2a2-1a94c4ac9427 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:53.138217 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 649/1302] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:18:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:18:53.143900 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d2ff2-2777-4b58-a725-34cbe6882dc6 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:53.152666 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d2ff2-2777-4b58-a725-34cbe6882dc6 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:53.152666 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d2ff2-2777-4b58-a725-34cbe6882dc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:53.160920 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-2b2d2ff2-2777-4b58-a725-34cbe6882dc6 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:18:53.162149 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-2b2d2ff2-2777-4b58-a725-34cbe6882dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:18:53.230543 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 654/1303] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:18:53 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:18:53.254127 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53fd43c-5cf3-4458-a2a2-1a94c4ac9427 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:53.350306 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:53.357351 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53fd43c-5cf3-4458-a2a2-1a94c4ac9427 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:53.391168 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0085b11b-0d09-461a-9e96-880a6beda92b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:53.391884 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0085b11b-0d09-461a-9e96-880a6beda92b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:53.392161 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0085b11b-0d09-461a-9e96-880a6beda92b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:53.392431 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0085b11b-0d09-461a-9e96-880a6beda92b None admin] PATH_INFO: `/v3/services/placement` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:53.393067 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0085b11b-0d09-461a-9e96-880a6beda92b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:53.394631 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0085b11b-0d09-461a-9e96-880a6beda92b None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:53.406926 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-0085b11b-0d09-461a-9e96-880a6beda92b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 15 19:18:53.406926 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:18:53.406926 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 19:18:53.406926 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 19:18:53.406926 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 19:18:53.406926 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 19:18:53.406926 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 19:18:53.406926 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:18:53.406926 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 15 19:18:53.406926 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:18:53.406926 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 19:18:53.406926 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:18:53.406926 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:18:53.406926 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:18:53.406926 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:18:53.406926 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:18:53.406926 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:18:53.412210 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:18:53.412210 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:18:53.412210 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:18:53.412210 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:18:53.412210 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:18:53.412210 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:18:53.412210 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 19:18:53.412210 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 19:18:53.412210 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 19:18:53.412210 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 19:18:53.412210 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:18:53.412210 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:18:53.412210 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:18:53.412210 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:18:53.412210 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:18:53.412210 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:18:53.413407 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:18:53.413407 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 19:18:53.413407 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:18:53.413407 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:18:53.413407 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:18:53.413407 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:18:53.413407 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:18:53.413407 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:18:53.413407 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:18:53.413407 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:18:53.413407 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 19:18:53.413407 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:18:53.413407 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 15 19:18:53.413407 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 19:18:53.413407 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 650/1304] 10.222.0.33 () {56 vars in 1117 bytes} [Fri Jul 15 19:18:53 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:18:53.414430 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2d2ff2-2777-4b58-a725-34cbe6882dc6 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:53.491168 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:53.513200 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2d2ff2-2777-4b58-a725-34cbe6882dc6 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:53.547104 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589f1cf4-c22d-48a3-86fd-29487477d035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:53.547763 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f1cf4-c22d-48a3-86fd-29487477d035 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:53.548045 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f1cf4-c22d-48a3-86fd-29487477d035 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:53.548320 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f1cf4-c22d-48a3-86fd-29487477d035 None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:53.549025 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589f1cf4-c22d-48a3-86fd-29487477d035 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:18:53.549328 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589f1cf4-c22d-48a3-86fd-29487477d035 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:53.550776 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589f1cf4-c22d-48a3-86fd-29487477d035 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:53.555952 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 655/1305] 10.222.0.33 () {56 vars in 1116 bytes} [Fri Jul 15 19:18:53 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:18:53.561755 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0085b11b-0d09-461a-9e96-880a6beda92b None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:53.661183 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0085b11b-0d09-461a-9e96-880a6beda92b None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:53.694247 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5002074-0977-431c-8112-eac36e4b87cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:53.694899 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5002074-0977-431c-8112-eac36e4b87cf None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:53.695182 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5002074-0977-431c-8112-eac36e4b87cf None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:53.695458 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5002074-0977-431c-8112-eac36e4b87cf None admin] PATH_INFO: `/v3/endpoints` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:53.696204 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5002074-0977-431c-8112-eac36e4b87cf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:53.697574 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5002074-0977-431c-8112-eac36e4b87cf None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:53.774701 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 651/1306] 10.222.0.33 () {60 vars in 1144 bytes} [Fri Jul 15 19:18:53 2022] POST /identity/v3/endpoints => generated 331 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:19:03.585998 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=30 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:19:03.769909 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:26:59.090377 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2334ec7b-6700-4d84-9585-51b6b8973276 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:59.090920 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2334ec7b-6700-4d84-9585-51b6b8973276 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:59.090920 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2334ec7b-6700-4d84-9585-51b6b8973276 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:59.098023 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8bb410-b52d-444f-8f9e-fb21f24d728a None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:59.098459 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8bb410-b52d-444f-8f9e-fb21f24d728a None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:59.098684 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8bb410-b52d-444f-8f9e-fb21f24d728a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:59.102165 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 656/1307] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:26:59 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:26:59.173289 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-1b8bb410-b52d-444f-8f9e-fb21f24d728a None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:26:59.174546 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-1b8bb410-b52d-444f-8f9e-fb21f24d728a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:26:59.299128 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 652/1308] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:26:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:26:59.337624 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca19ec47-b73c-4da0-9675-8773a2d77632 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:59.337946 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca19ec47-b73c-4da0-9675-8773a2d77632 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:59.338224 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca19ec47-b73c-4da0-9675-8773a2d77632 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:59.340010 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 657/1309] 10.222.0.33 () {54 vars in 964 bytes} [Fri Jul 15 19:26:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:26:59.347191 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d31598-54e7-44dc-919d-6f9d9d3d3419 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:59.347564 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d31598-54e7-44dc-919d-6f9d9d3d3419 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:59.347852 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d31598-54e7-44dc-919d-6f9d9d3d3419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:59.434478 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-42d31598-54e7-44dc-919d-6f9d9d3d3419 None None] MFA Rules not processed for user `b2a283d7d3594e37943bda4529d0a775`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:26:59.435459 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-42d31598-54e7-44dc-919d-6f9d9d3d3419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:26:59.465146 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:26:59.540250 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 653/1310] 10.222.0.33 () {58 vars in 1061 bytes} [Fri Jul 15 19:26:59 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:26:59.549078 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca19ec47-b73c-4da0-9675-8773a2d77632 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:59.551583 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-ca19ec47-b73c-4da0-9675-8773a2d77632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:26:59.676882 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca19ec47-b73c-4da0-9675-8773a2d77632 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:59.759911 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003abfda-324b-45b4-9d7a-430e8806c532 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:59.760500 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003abfda-324b-45b4-9d7a-430e8806c532 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:59.760777 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003abfda-324b-45b4-9d7a-430e8806c532 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:59.761050 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003abfda-324b-45b4-9d7a-430e8806c532 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:59.762747 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-003abfda-324b-45b4-9d7a-430e8806c532 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:26:59.821331 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:26:59.880093 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003abfda-324b-45b4-9d7a-430e8806c532 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:59.882625 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003abfda-324b-45b4-9d7a-430e8806c532 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:59.902949 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 658/1311] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:26:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 19:27:02.262485 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0839323e-6845-4686-bcf6-c639fcab39c0 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:02.263044 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0839323e-6845-4686-bcf6-c639fcab39c0 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:02.263044 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0839323e-6845-4686-bcf6-c639fcab39c0 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:02.264945 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 654/1312] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:27:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:27:02.270041 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91d9f9c-b9aa-4d45-9eac-f22e3a9c8043 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:02.270486 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91d9f9c-b9aa-4d45-9eac-f22e3a9c8043 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:02.270596 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91d9f9c-b9aa-4d45-9eac-f22e3a9c8043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:02.352827 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-f91d9f9c-b9aa-4d45-9eac-f22e3a9c8043 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:27:02.354387 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-f91d9f9c-b9aa-4d45-9eac-f22e3a9c8043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:02.451127 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 659/1313] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:27:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:27:02.506021 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4dfefd-7605-40fa-a15c-a02f7cdc2295 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:02.506403 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4dfefd-7605-40fa-a15c-a02f7cdc2295 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:02.506798 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4dfefd-7605-40fa-a15c-a02f7cdc2295 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:02.508246 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 655/1314] 10.222.0.33 () {54 vars in 964 bytes} [Fri Jul 15 19:27:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:27:02.515414 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24021e2-acd8-4f99-86c2-7ea3b0f726cf None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:02.515645 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24021e2-acd8-4f99-86c2-7ea3b0f726cf None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:02.515920 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24021e2-acd8-4f99-86c2-7ea3b0f726cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:02.526870 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:27:02.591124 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-b24021e2-acd8-4f99-86c2-7ea3b0f726cf None None] MFA Rules not processed for user `b2a283d7d3594e37943bda4529d0a775`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:27:02.592343 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-b24021e2-acd8-4f99-86c2-7ea3b0f726cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:02.677783 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 660/1315] 10.222.0.33 () {58 vars in 1061 bytes} [Fri Jul 15 19:27:02 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:27:02.687567 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4dfefd-7605-40fa-a15c-a02f7cdc2295 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:02.689142 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-2b4dfefd-7605-40fa-a15c-a02f7cdc2295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:02.803440 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4dfefd-7605-40fa-a15c-a02f7cdc2295 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:02.874500 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:27:02.877085 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621fefd0-3c7f-4799-9340-5e4ed25ec15a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:02.877736 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621fefd0-3c7f-4799-9340-5e4ed25ec15a service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:02.878061 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621fefd0-3c7f-4799-9340-5e4ed25ec15a service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:02.878338 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621fefd0-3c7f-4799-9340-5e4ed25ec15a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:02.880023 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-621fefd0-3c7f-4799-9340-5e4ed25ec15a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:02.994678 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621fefd0-3c7f-4799-9340-5e4ed25ec15a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:02.996486 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621fefd0-3c7f-4799-9340-5e4ed25ec15a service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:03.015878 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 656/1316] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:27:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:27:05.813243 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ea4b29-0c02-4bb9-82b4-cb2f7e757cb1 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:05.813782 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ea4b29-0c02-4bb9-82b4-cb2f7e757cb1 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:05.813906 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ea4b29-0c02-4bb9-82b4-cb2f7e757cb1 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:05.815285 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 661/1317] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:27:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:27:05.820487 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965c5b0e-256e-4bd9-a00c-3d86c608762f None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:05.820914 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965c5b0e-256e-4bd9-a00c-3d86c608762f None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:05.821127 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965c5b0e-256e-4bd9-a00c-3d86c608762f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:05.884764 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-965c5b0e-256e-4bd9-a00c-3d86c608762f None None] MFA Rules not processed for user `da619d0bd9ad4a48a65b481207b21979`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:27:05.886070 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-965c5b0e-256e-4bd9-a00c-3d86c608762f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:05.937092 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:27:05.981104 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 657/1318] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:27:05 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:27:06.020585 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ea4b29-0c02-4bb9-82b4-cb2f7e757cb1 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:06.022209 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-74ea4b29-0c02-4bb9-82b4-cb2f7e757cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:06.124722 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ea4b29-0c02-4bb9-82b4-cb2f7e757cb1 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:06.127489 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:27:06.196517 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e79e2c-5935-4434-9720-78b9f25eb0e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:06.197158 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e79e2c-5935-4434-9720-78b9f25eb0e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:06.197442 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e79e2c-5935-4434-9720-78b9f25eb0e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:06.197718 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e79e2c-5935-4434-9720-78b9f25eb0e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:06.199381 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-a5e79e2c-5935-4434-9720-78b9f25eb0e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:06.304468 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e79e2c-5935-4434-9720-78b9f25eb0e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:06.305923 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e79e2c-5935-4434-9720-78b9f25eb0e9 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:06.324196 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 662/1319] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:27:06 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:27:09.174464 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507a7591-9907-47d4-b35b-0aa47894aa81 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:09.174988 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507a7591-9907-47d4-b35b-0aa47894aa81 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:09.174988 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507a7591-9907-47d4-b35b-0aa47894aa81 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:09.176310 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 658/1320] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:27:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:27:09.181552 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec330b5-45a5-4c35-b51a-aa0d4bdbf958 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:09.182086 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec330b5-45a5-4c35-b51a-aa0d4bdbf958 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:09.182192 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec330b5-45a5-4c35-b51a-aa0d4bdbf958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:09.221314 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:27:09.245320 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-aec330b5-45a5-4c35-b51a-aa0d4bdbf958 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:27:09.246565 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-aec330b5-45a5-4c35-b51a-aa0d4bdbf958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:09.346141 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 663/1321] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:27:09 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:27:09.384814 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507a7591-9907-47d4-b35b-0aa47894aa81 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:09.386430 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-507a7591-9907-47d4-b35b-0aa47894aa81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:09.489524 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507a7591-9907-47d4-b35b-0aa47894aa81 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:09.562249 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5840b2-e971-4dee-8da1-9448e53e1d23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:09.562896 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5840b2-e971-4dee-8da1-9448e53e1d23 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:09.563175 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5840b2-e971-4dee-8da1-9448e53e1d23 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:09.563449 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5840b2-e971-4dee-8da1-9448e53e1d23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:09.565100 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-7f5840b2-e971-4dee-8da1-9448e53e1d23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:09.572491 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:27:09.671793 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5840b2-e971-4dee-8da1-9448e53e1d23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:09.673220 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5840b2-e971-4dee-8da1-9448e53e1d23 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:09.691830 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 659/1322] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:27:09 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:27:12.839734 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48710e20-fc8b-43d4-958a-55d324877826 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:12.840284 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48710e20-fc8b-43d4-958a-55d324877826 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:12.840284 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48710e20-fc8b-43d4-958a-55d324877826 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:12.841642 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 664/1323] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:27:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:27:12.846850 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01142e1-738d-4a31-9501-9e65c37f3b25 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:12.847260 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01142e1-738d-4a31-9501-9e65c37f3b25 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:12.847404 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01142e1-738d-4a31-9501-9e65c37f3b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:12.912299 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-e01142e1-738d-4a31-9501-9e65c37f3b25 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:27:12.913488 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e01142e1-738d-4a31-9501-9e65c37f3b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:12.971316 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:27:13.009532 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 660/1324] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:27:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:27:13.048185 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48710e20-fc8b-43d4-958a-55d324877826 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:13.049843 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-48710e20-fc8b-43d4-958a-55d324877826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:13.161034 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48710e20-fc8b-43d4-958a-55d324877826 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:13.176038 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:27:13.243234 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6514c5a3-eb84-4095-b596-3b650850b7ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:13.243893 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6514c5a3-eb84-4095-b596-3b650850b7ea service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:13.244175 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6514c5a3-eb84-4095-b596-3b650850b7ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:13.244446 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6514c5a3-eb84-4095-b596-3b650850b7ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:13.246170 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-6514c5a3-eb84-4095-b596-3b650850b7ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:13.364020 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6514c5a3-eb84-4095-b596-3b650850b7ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:13.365454 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6514c5a3-eb84-4095-b596-3b650850b7ea service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:13.384118 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 665/1325] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:27:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:27:15.464773 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1406f004-8922-4a7b-accd-a38dce2dd77b None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:15.465339 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1406f004-8922-4a7b-accd-a38dce2dd77b None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:15.465339 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1406f004-8922-4a7b-accd-a38dce2dd77b None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:15.466743 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 661/1326] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:27:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:27:15.472156 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b680c585-694b-42a9-84c8-f7c0af3c14c1 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:15.472614 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b680c585-694b-42a9-84c8-f7c0af3c14c1 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:15.472828 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b680c585-694b-42a9-84c8-f7c0af3c14c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:15.520973 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:27:15.539340 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-b680c585-694b-42a9-84c8-f7c0af3c14c1 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:27:15.540584 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-b680c585-694b-42a9-84c8-f7c0af3c14c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:15.638134 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 666/1327] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:27:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:27:15.675816 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1406f004-8922-4a7b-accd-a38dce2dd77b None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:15.677438 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-1406f004-8922-4a7b-accd-a38dce2dd77b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:15.783412 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1406f004-8922-4a7b-accd-a38dce2dd77b None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:15.860968 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76ff8bb-15c7-44ff-981d-c46eee91dff0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:15.861621 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76ff8bb-15c7-44ff-981d-c46eee91dff0 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:15.861941 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76ff8bb-15c7-44ff-981d-c46eee91dff0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:15.862217 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76ff8bb-15c7-44ff-981d-c46eee91dff0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:15.863911 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-a76ff8bb-15c7-44ff-981d-c46eee91dff0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:15.883262 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:27:15.978861 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76ff8bb-15c7-44ff-981d-c46eee91dff0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:15.980228 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76ff8bb-15c7-44ff-981d-c46eee91dff0 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:15.999613 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 662/1328] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:27:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:27:18.068157 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be05af52-66f9-4a07-b33d-9dbc2a80c340 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:18.068684 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be05af52-66f9-4a07-b33d-9dbc2a80c340 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:18.068684 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be05af52-66f9-4a07-b33d-9dbc2a80c340 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:18.070158 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 667/1329] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:27:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:27:18.075503 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d993609-d607-49f8-8aec-d8575a70dc89 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:18.075918 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d993609-d607-49f8-8aec-d8575a70dc89 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:18.076069 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d993609-d607-49f8-8aec-d8575a70dc89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:18.143193 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-8d993609-d607-49f8-8aec-d8575a70dc89 None None] MFA Rules not processed for user `da619d0bd9ad4a48a65b481207b21979`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:27:18.144414 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-8d993609-d607-49f8-8aec-d8575a70dc89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:18.212463 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:27:18.244079 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 663/1330] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:27:18 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:27:18.283551 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be05af52-66f9-4a07-b33d-9dbc2a80c340 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:18.285078 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-be05af52-66f9-4a07-b33d-9dbc2a80c340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:18.389227 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be05af52-66f9-4a07-b33d-9dbc2a80c340 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:18.409880 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:27:18.465378 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32487f4-9bc0-4d22-b70a-0440312c8d5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:18.466083 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32487f4-9bc0-4d22-b70a-0440312c8d5d service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:18.466374 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32487f4-9bc0-4d22-b70a-0440312c8d5d service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:18.466652 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32487f4-9bc0-4d22-b70a-0440312c8d5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:18.469004 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-f32487f4-9bc0-4d22-b70a-0440312c8d5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:18.580663 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32487f4-9bc0-4d22-b70a-0440312c8d5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:18.582136 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32487f4-9bc0-4d22-b70a-0440312c8d5d service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:18.600516 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 668/1331] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:27:18 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:27:21.325859 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240517ff-ab0c-4761-b535-3d91b1c676bf None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:21.326414 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240517ff-ab0c-4761-b535-3d91b1c676bf None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:21.326414 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240517ff-ab0c-4761-b535-3d91b1c676bf None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:21.327784 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 664/1332] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:27:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:27:21.332915 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbfc786-1ecb-4f0c-9c5e-abf570cd4eb2 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:21.333361 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbfc786-1ecb-4f0c-9c5e-abf570cd4eb2 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:21.333460 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbfc786-1ecb-4f0c-9c5e-abf570cd4eb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:21.395737 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:27:21.398626 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-7cbfc786-1ecb-4f0c-9c5e-abf570cd4eb2 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:27:21.399862 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-7cbfc786-1ecb-4f0c-9c5e-abf570cd4eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:21.495004 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 669/1333] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:27:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:27:21.531426 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240517ff-ab0c-4761-b535-3d91b1c676bf None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:21.533005 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-240517ff-ab0c-4761-b535-3d91b1c676bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:21.634553 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240517ff-ab0c-4761-b535-3d91b1c676bf None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:21.707319 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0390400-445f-4238-9928-38aae072bd7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:21.707968 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0390400-445f-4238-9928-38aae072bd7a service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:21.708247 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0390400-445f-4238-9928-38aae072bd7a service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:21.708523 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0390400-445f-4238-9928-38aae072bd7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:21.710215 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d0390400-445f-4238-9928-38aae072bd7a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:21.737236 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:27:21.818411 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0390400-445f-4238-9928-38aae072bd7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:21.819858 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0390400-445f-4238-9928-38aae072bd7a service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:21.839783 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 665/1334] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:27:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:27:23.950655 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54a404f-4765-4c67-8ed5-a289eacea0c5 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:23.951228 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54a404f-4765-4c67-8ed5-a289eacea0c5 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:23.951328 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54a404f-4765-4c67-8ed5-a289eacea0c5 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:23.952669 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 670/1335] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:27:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:27:23.957854 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7129002-56b3-4862-9c85-5a5264cc7a29 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:23.958257 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7129002-56b3-4862-9c85-5a5264cc7a29 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:23.958401 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7129002-56b3-4862-9c85-5a5264cc7a29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:24.021889 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-a7129002-56b3-4862-9c85-5a5264cc7a29 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:27:24.023332 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-a7129002-56b3-4862-9c85-5a5264cc7a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:24.096481 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:27:24.118791 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 666/1336] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:27:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:27:24.159270 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54a404f-4765-4c67-8ed5-a289eacea0c5 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:24.160830 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-e54a404f-4765-4c67-8ed5-a289eacea0c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:24.270296 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54a404f-4765-4c67-8ed5-a289eacea0c5 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:24.299233 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:27:24.343341 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74796e6d-a3db-4a22-b879-0ccad34f27e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:24.343982 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74796e6d-a3db-4a22-b879-0ccad34f27e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:24.344267 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74796e6d-a3db-4a22-b879-0ccad34f27e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:24.344545 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74796e6d-a3db-4a22-b879-0ccad34f27e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:24.346231 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-74796e6d-a3db-4a22-b879-0ccad34f27e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:24.451849 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74796e6d-a3db-4a22-b879-0ccad34f27e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:24.453204 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74796e6d-a3db-4a22-b879-0ccad34f27e7 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:24.472016 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 671/1337] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:27:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:27:27.245744 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8480ce4c-5562-4e14-90eb-0f6cb719abd6 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:27.246300 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8480ce4c-5562-4e14-90eb-0f6cb719abd6 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:27.246300 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8480ce4c-5562-4e14-90eb-0f6cb719abd6 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:27.247639 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 667/1338] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:27:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:27:27.253567 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681a43c3-34c7-4b08-94ee-e8f3e2d1c554 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:27.254018 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681a43c3-34c7-4b08-94ee-e8f3e2d1c554 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:27.254162 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681a43c3-34c7-4b08-94ee-e8f3e2d1c554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:27.321096 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-681a43c3-34c7-4b08-94ee-e8f3e2d1c554 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:27:27.322363 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-681a43c3-34c7-4b08-94ee-e8f3e2d1c554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:27.331146 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:27:27.417311 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 672/1339] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:27:27 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:27:27.453898 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8480ce4c-5562-4e14-90eb-0f6cb719abd6 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:27.455453 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-8480ce4c-5562-4e14-90eb-0f6cb719abd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:27.557198 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8480ce4c-5562-4e14-90eb-0f6cb719abd6 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:27.630444 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f5b1ae-9bb4-42c2-96d0-15c4aaf76a58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:27.631093 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f5b1ae-9bb4-42c2-96d0-15c4aaf76a58 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:27.631385 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f5b1ae-9bb4-42c2-96d0-15c4aaf76a58 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:27.631657 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f5b1ae-9bb4-42c2-96d0-15c4aaf76a58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:27.633321 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-26f5b1ae-9bb4-42c2-96d0-15c4aaf76a58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:27.668101 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:27:27.741078 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f5b1ae-9bb4-42c2-96d0-15c4aaf76a58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:27.742546 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f5b1ae-9bb4-42c2-96d0-15c4aaf76a58 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:27.762665 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 668/1340] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:27:27 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:27:32.038116 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247a5fa3-fc03-49dc-8f07-e3e9376b4d14 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:32.038577 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247a5fa3-fc03-49dc-8f07-e3e9376b4d14 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:32.038814 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247a5fa3-fc03-49dc-8f07-e3e9376b4d14 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:32.040510 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 673/1341] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:27:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:27:32.046281 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b94c941-17c7-4fb9-be09-3b76361ebae1 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:32.046759 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b94c941-17c7-4fb9-be09-3b76361ebae1 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:32.046865 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b94c941-17c7-4fb9-be09-3b76361ebae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:32.114937 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-9b94c941-17c7-4fb9-be09-3b76361ebae1 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:27:32.116282 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9b94c941-17c7-4fb9-be09-3b76361ebae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:32.209058 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:27:32.223326 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 669/1342] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:27:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:27:32.269248 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247a5fa3-fc03-49dc-8f07-e3e9376b4d14 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:32.270860 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-247a5fa3-fc03-49dc-8f07-e3e9376b4d14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:32.377086 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247a5fa3-fc03-49dc-8f07-e3e9376b4d14 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:32.415549 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:27:32.452444 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa921dc-a04c-41a8-94c0-2d8829a2bc32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:32.453356 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa921dc-a04c-41a8-94c0-2d8829a2bc32 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:32.453856 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa921dc-a04c-41a8-94c0-2d8829a2bc32 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:32.454458 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa921dc-a04c-41a8-94c0-2d8829a2bc32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:32.456127 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-5aa921dc-a04c-41a8-94c0-2d8829a2bc32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:32.566175 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa921dc-a04c-41a8-94c0-2d8829a2bc32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:32.567728 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa921dc-a04c-41a8-94c0-2d8829a2bc32 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:32.587056 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 674/1343] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:27:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 19:27:35.514938 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a16444-ce80-432a-adf9-1fac491fab2f None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:35.515357 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a16444-ce80-432a-adf9-1fac491fab2f None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:35.515496 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a16444-ce80-432a-adf9-1fac491fab2f None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:35.516891 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 670/1344] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:27:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 19:27:35.522502 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18420676-efe6-47ad-b9c4-114cfbe0db4c None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:35.522891 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18420676-efe6-47ad-b9c4-114cfbe0db4c None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:35.523083 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18420676-efe6-47ad-b9c4-114cfbe0db4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:35.590415 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-18420676-efe6-47ad-b9c4-114cfbe0db4c None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:27:35.591687 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-18420676-efe6-47ad-b9c4-114cfbe0db4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:35.611232 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:27:35.691865 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 675/1345] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:27:35 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:27:35.737264 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a16444-ce80-432a-adf9-1fac491fab2f None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:35.742202 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-c3a16444-ce80-432a-adf9-1fac491fab2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:35.865906 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a16444-ce80-432a-adf9-1fac491fab2f None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:35.957270 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396ddd58-ed4f-4e7c-93af-a924ac6dccae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:35.957961 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ddd58-ed4f-4e7c-93af-a924ac6dccae service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:35.958243 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ddd58-ed4f-4e7c-93af-a924ac6dccae service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:35.958514 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ddd58-ed4f-4e7c-93af-a924ac6dccae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:35.960175 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-396ddd58-ed4f-4e7c-93af-a924ac6dccae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:36.005138 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:27:36.075338 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396ddd58-ed4f-4e7c-93af-a924ac6dccae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:36.076711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396ddd58-ed4f-4e7c-93af-a924ac6dccae service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:36.095770 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 671/1346] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:27:35 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:27:40.757441 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34656ee-bb60-47a2-b9cc-6a5f185dbcb7 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:40.758030 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34656ee-bb60-47a2-b9cc-6a5f185dbcb7 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:40.758030 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34656ee-bb60-47a2-b9cc-6a5f185dbcb7 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:40.759439 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 676/1347] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:27:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:27:40.765569 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dfd091-b8ff-4a99-87c5-e6070790e477 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:40.766141 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dfd091-b8ff-4a99-87c5-e6070790e477 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:40.766238 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dfd091-b8ff-4a99-87c5-e6070790e477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:40.837623 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-67dfd091-b8ff-4a99-87c5-e6070790e477 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:27:40.838872 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-67dfd091-b8ff-4a99-87c5-e6070790e477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:40.937061 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:27:40.939735 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 672/1348] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:27:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:27:40.978159 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34656ee-bb60-47a2-b9cc-6a5f185dbcb7 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:40.979694 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-b34656ee-bb60-47a2-b9cc-6a5f185dbcb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:41.096514 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34656ee-bb60-47a2-b9cc-6a5f185dbcb7 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:41.156976 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:27:41.183083 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22c036e-e751-4b85-bd7d-8998407a0d11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:41.183746 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22c036e-e751-4b85-bd7d-8998407a0d11 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:41.184089 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22c036e-e751-4b85-bd7d-8998407a0d11 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:41.184362 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22c036e-e751-4b85-bd7d-8998407a0d11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:41.186604 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-c22c036e-e751-4b85-bd7d-8998407a0d11 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:41.301304 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22c036e-e751-4b85-bd7d-8998407a0d11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:41.302778 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22c036e-e751-4b85-bd7d-8998407a0d11 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:41.321960 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 677/1349] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:27:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:27:46.168173 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a749ec-df17-42c2-a22f-6ec8eb2098c2 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:46.168711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a749ec-df17-42c2-a22f-6ec8eb2098c2 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:46.169017 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a749ec-df17-42c2-a22f-6ec8eb2098c2 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:46.173013 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 673/1350] 10.222.0.33 () {54 vars in 921 bytes} [Fri Jul 15 19:27:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:27:46.179833 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a916bd-efda-466f-a227-2b61ae125b1e None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:46.180896 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a916bd-efda-466f-a227-2b61ae125b1e None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:46.181428 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a916bd-efda-466f-a227-2b61ae125b1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:46.269811 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-05a916bd-efda-466f-a227-2b61ae125b1e None None] MFA Rules not processed for user `52a0fce231ba4785a685ac9898d07503`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:27:46.271127 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-05a916bd-efda-466f-a227-2b61ae125b1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:46.321134 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:27:46.430457 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 678/1351] 10.222.0.33 () {58 vars in 1018 bytes} [Fri Jul 15 19:27:46 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:27:50.575736 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490955c4-69bb-435a-85a3-ace5b760a82b None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:50.576643 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490955c4-69bb-435a-85a3-ace5b760a82b None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:50.576920 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490955c4-69bb-435a-85a3-ace5b760a82b None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:50.578509 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 674/1352] 10.222.0.33 () {54 vars in 921 bytes} [Fri Jul 15 19:27:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:27:50.584316 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dbfa2e-3b40-4e8d-901a-eb2b25324be0 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:50.585151 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dbfa2e-3b40-4e8d-901a-eb2b25324be0 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:50.585664 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dbfa2e-3b40-4e8d-901a-eb2b25324be0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:50.667275 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-34dbfa2e-3b40-4e8d-901a-eb2b25324be0 None None] MFA Rules not processed for user `52a0fce231ba4785a685ac9898d07503`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:27:50.668859 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-34dbfa2e-3b40-4e8d-901a-eb2b25324be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:50.746347 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:27:50.790724 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 679/1353] 10.222.0.33 () {58 vars in 1018 bytes} [Fri Jul 15 19:27:50 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:27:52.121085 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2de0384-ad6e-445d-826f-10ccf3644f14 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:52.121654 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2de0384-ad6e-445d-826f-10ccf3644f14 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:52.121654 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2de0384-ad6e-445d-826f-10ccf3644f14 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:52.123546 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 675/1354] 10.222.0.33 () {54 vars in 921 bytes} [Fri Jul 15 19:27:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:27:52.128807 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34045766-1aa8-4057-b8f3-d749716452a7 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:52.129226 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34045766-1aa8-4057-b8f3-d749716452a7 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:52.129361 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34045766-1aa8-4057-b8f3-d749716452a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:52.202497 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-34045766-1aa8-4057-b8f3-d749716452a7 None None] MFA Rules not processed for user `52a0fce231ba4785a685ac9898d07503`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:27:52.203690 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-34045766-1aa8-4057-b8f3-d749716452a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:52.311222 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 680/1355] 10.222.0.33 () {58 vars in 1018 bytes} [Fri Jul 15 19:27:52 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:27:55.899476 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bd88ca-ea30-45c9-afb0-63b562ae349b None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:55.899978 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bd88ca-ea30-45c9-afb0-63b562ae349b None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:55.900089 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bd88ca-ea30-45c9-afb0-63b562ae349b None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:55.901587 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 676/1356] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:27:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:27:55.907209 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fe2803-7d43-4509-ad4b-5b0eb68539cd None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:55.907674 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fe2803-7d43-4509-ad4b-5b0eb68539cd None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:55.908082 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fe2803-7d43-4509-ad4b-5b0eb68539cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:55.945899 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:27:55.981470 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-f1fe2803-7d43-4509-ad4b-5b0eb68539cd None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:27:55.982875 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-f1fe2803-7d43-4509-ad4b-5b0eb68539cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:56.089884 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 681/1357] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:27:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:27:56.146557 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f75822-1ccc-4cad-bf62-9a595ea857b7 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:56.147481 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f75822-1ccc-4cad-bf62-9a595ea857b7 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:56.147960 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f75822-1ccc-4cad-bf62-9a595ea857b7 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:56.152509 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 677/1358] 10.222.0.33 () {54 vars in 961 bytes} [Fri Jul 15 19:27:56 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:27:56.160015 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a488b26-d227-413c-b984-2a5d929ea3e4 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:56.163408 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a488b26-d227-413c-b984-2a5d929ea3e4 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:56.163408 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a488b26-d227-413c-b984-2a5d929ea3e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:56.249874 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-2a488b26-d227-413c-b984-2a5d929ea3e4 None None] MFA Rules not processed for user `7673a3b5c0324e07a9535ac70ed7a491`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:27:56.251219 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-2a488b26-d227-413c-b984-2a5d929ea3e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:56.307478 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:27:56.377586 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 682/1359] 10.222.0.33 () {58 vars in 1058 bytes} [Fri Jul 15 19:27:56 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:27:56.385203 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f75822-1ccc-4cad-bf62-9a595ea857b7 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:56.386754 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e7f75822-1ccc-4cad-bf62-9a595ea857b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:56.539396 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f75822-1ccc-4cad-bf62-9a595ea857b7 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:56.618484 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:27:56.667368 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caebf084-5299-4e78-a98f-0b0da65fc743 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7673a3b5c0324e07a9535ac70ed7a491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:56.669394 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caebf084-5299-4e78-a98f-0b0da65fc743 service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:56.669394 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caebf084-5299-4e78-a98f-0b0da65fc743 service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:56.669701 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caebf084-5299-4e78-a98f-0b0da65fc743 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:56.671495 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-caebf084-5299-4e78-a98f-0b0da65fc743 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:56.770889 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caebf084-5299-4e78-a98f-0b0da65fc743 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:56.775803 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caebf084-5299-4e78-a98f-0b0da65fc743 service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:56.792147 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 678/1360] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:27:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:27:59.037324 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e4e11d-d284-442f-8a45-91e3f3586530 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:59.038043 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e4e11d-d284-442f-8a45-91e3f3586530 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:59.038043 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e4e11d-d284-442f-8a45-91e3f3586530 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:59.039525 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 683/1361] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:27:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:27:59.045668 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb2d248-bc78-4a5e-be13-c176296254f1 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:59.046487 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb2d248-bc78-4a5e-be13-c176296254f1 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:59.046487 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb2d248-bc78-4a5e-be13-c176296254f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:59.117540 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-abb2d248-bc78-4a5e-be13-c176296254f1 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:27:59.118792 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-abb2d248-bc78-4a5e-be13-c176296254f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:59.158701 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:27:59.225862 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 679/1362] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:27:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:27:59.271277 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e4e11d-d284-442f-8a45-91e3f3586530 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:59.273386 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-32e4e11d-d284-442f-8a45-91e3f3586530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:59.449625 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e4e11d-d284-442f-8a45-91e3f3586530 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:59.459611 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:27:59.616650 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb6abcb-9538-4b6b-b370-506258e28456 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7673a3b5c0324e07a9535ac70ed7a491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:59.617453 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6abcb-9538-4b6b-b370-506258e28456 service nova] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:59.617884 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6abcb-9538-4b6b-b370-506258e28456 service nova] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:59.618290 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6abcb-9538-4b6b-b370-506258e28456 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:59.620104 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-7cb6abcb-9538-4b6b-b370-506258e28456 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:59.720321 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb6abcb-9538-4b6b-b370-506258e28456 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:59.722106 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb6abcb-9538-4b6b-b370-506258e28456 service nova] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:59.742550 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 684/1363] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:27:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:28:02.205802 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298baaff-b1e2-4b5f-8d16-a9699e0e6b56 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:02.206701 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298baaff-b1e2-4b5f-8d16-a9699e0e6b56 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:02.206981 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298baaff-b1e2-4b5f-8d16-a9699e0e6b56 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:02.208402 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 680/1364] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:28:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:28:02.214031 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a450cf-a3ca-4f43-8de5-0260af05273e None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:02.214481 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a450cf-a3ca-4f43-8de5-0260af05273e None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:02.214600 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a450cf-a3ca-4f43-8de5-0260af05273e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:02.232316 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:28:02.286364 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-24a450cf-a3ca-4f43-8de5-0260af05273e None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:28:02.287582 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-24a450cf-a3ca-4f43-8de5-0260af05273e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:02.391320 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 685/1365] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:28:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:28:02.439706 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efffe87b-a010-4604-a10b-9f7770b193ae None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:02.440227 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efffe87b-a010-4604-a10b-9f7770b193ae None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:02.440551 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efffe87b-a010-4604-a10b-9f7770b193ae None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:02.441975 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 681/1366] 10.222.0.33 () {54 vars in 961 bytes} [Fri Jul 15 19:28:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:28:02.447540 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48dec8a-9423-4331-89f5-e0acb0edb9a7 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:02.447831 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48dec8a-9423-4331-89f5-e0acb0edb9a7 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:02.448111 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48dec8a-9423-4331-89f5-e0acb0edb9a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:02.538746 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-e48dec8a-9423-4331-89f5-e0acb0edb9a7 None None] MFA Rules not processed for user `7673a3b5c0324e07a9535ac70ed7a491`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:28:02.546166 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-e48dec8a-9423-4331-89f5-e0acb0edb9a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:02.584029 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:28:02.665326 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 686/1367] 10.222.0.33 () {58 vars in 1058 bytes} [Fri Jul 15 19:28:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:28:02.674033 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efffe87b-a010-4604-a10b-9f7770b193ae None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:28:02.675602 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-efffe87b-a010-4604-a10b-9f7770b193ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:02.831404 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efffe87b-a010-4604-a10b-9f7770b193ae None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:28:02.855957 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:28:02.956702 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fd8e20-d5f5-4c16-833e-ab4dbeff7c2c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7673a3b5c0324e07a9535ac70ed7a491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:28:02.957812 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fd8e20-d5f5-4c16-833e-ab4dbeff7c2c service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:02.958334 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fd8e20-d5f5-4c16-833e-ab4dbeff7c2c service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:02.958856 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fd8e20-d5f5-4c16-833e-ab4dbeff7c2c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:02.960739 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-09fd8e20-d5f5-4c16-833e-ab4dbeff7c2c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:03.058707 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fd8e20-d5f5-4c16-833e-ab4dbeff7c2c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:28:03.061251 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fd8e20-d5f5-4c16-833e-ab4dbeff7c2c service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:28:03.083972 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 682/1368] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:28:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:28:04.063846 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae34708-1562-4e1e-b63e-7ac1ef066c0f None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:04.064390 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae34708-1562-4e1e-b63e-7ac1ef066c0f None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:04.064390 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae34708-1562-4e1e-b63e-7ac1ef066c0f None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:04.065904 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 687/1369] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:28:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:28:04.071152 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9439f048-6b1c-4023-9409-0359da05e945 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:04.071638 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9439f048-6b1c-4023-9409-0359da05e945 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:04.071744 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9439f048-6b1c-4023-9409-0359da05e945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:04.104016 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:28:04.139832 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-9439f048-6b1c-4023-9409-0359da05e945 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:28:04.141068 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9439f048-6b1c-4023-9409-0359da05e945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:04.242390 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 683/1370] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:28:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:28:04.592419 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9303a5e-91b9-4b6e-ada2-0c555af4ad79 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:04.592698 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9303a5e-91b9-4b6e-ada2-0c555af4ad79 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:04.592976 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9303a5e-91b9-4b6e-ada2-0c555af4ad79 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:04.594573 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 688/1371] 10.222.0.33 () {54 vars in 963 bytes} [Fri Jul 15 19:28:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:28:04.600010 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4008868-ec22-4fc1-b09b-82ae22afb847 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:04.600478 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4008868-ec22-4fc1-b09b-82ae22afb847 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:04.600673 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4008868-ec22-4fc1-b09b-82ae22afb847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:04.681092 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-c4008868-ec22-4fc1-b09b-82ae22afb847 None None] MFA Rules not processed for user `c86c2df3d378490ca7ea497a73c19d40`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:28:04.682354 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-c4008868-ec22-4fc1-b09b-82ae22afb847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:04.739815 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:28:04.770660 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 684/1372] 10.222.0.33 () {58 vars in 1060 bytes} [Fri Jul 15 19:28:04 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:28:04.779696 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9303a5e-91b9-4b6e-ada2-0c555af4ad79 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:28:04.782041 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-a9303a5e-91b9-4b6e-ada2-0c555af4ad79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:04.899223 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9303a5e-91b9-4b6e-ada2-0c555af4ad79 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:28:04.941731 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:28:04.976378 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a96cc7-b568-48fa-8c0e-151af27cbfc5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86c2df3d378490ca7ea497a73c19d40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:28:04.977018 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a96cc7-b568-48fa-8c0e-151af27cbfc5 service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:04.977296 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a96cc7-b568-48fa-8c0e-151af27cbfc5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:04.977565 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a96cc7-b568-48fa-8c0e-151af27cbfc5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:04.979230 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-f9a96cc7-b568-48fa-8c0e-151af27cbfc5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:05.096361 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a96cc7-b568-48fa-8c0e-151af27cbfc5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:28:05.099895 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a96cc7-b568-48fa-8c0e-151af27cbfc5 service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:28:05.119912 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 689/1373] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:28:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:28:05.359075 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e32f11-5d9d-49d6-a9b3-3a134da779c3 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:05.359546 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e32f11-5d9d-49d6-a9b3-3a134da779c3 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:05.359644 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e32f11-5d9d-49d6-a9b3-3a134da779c3 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:05.361563 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 685/1374] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:28:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:28:05.366372 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da765254-f1cd-4b44-98aa-18da67cd1a8e None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:05.366860 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da765254-f1cd-4b44-98aa-18da67cd1a8e None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:05.366860 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da765254-f1cd-4b44-98aa-18da67cd1a8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:05.438956 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-da765254-f1cd-4b44-98aa-18da67cd1a8e None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:28:05.440049 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-da765254-f1cd-4b44-98aa-18da67cd1a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:05.464339 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:28:05.570060 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 690/1375] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:28:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:28:05.614240 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e32f11-5d9d-49d6-a9b3-3a134da779c3 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:28:05.615781 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d0e32f11-5d9d-49d6-a9b3-3a134da779c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:05.763751 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e32f11-5d9d-49d6-a9b3-3a134da779c3 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:28:05.820451 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:28:05.885635 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf34c2b-4810-4b16-ae46-a9654d643949 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7673a3b5c0324e07a9535ac70ed7a491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:28:05.886287 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf34c2b-4810-4b16-ae46-a9654d643949 service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:05.886573 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf34c2b-4810-4b16-ae46-a9654d643949 service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:05.887131 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf34c2b-4810-4b16-ae46-a9654d643949 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:05.889429 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0bf34c2b-4810-4b16-ae46-a9654d643949 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:05.980553 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf34c2b-4810-4b16-ae46-a9654d643949 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:28:05.982233 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf34c2b-4810-4b16-ae46-a9654d643949 service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:28:06.000642 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 686/1376] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:28:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:28:08.164817 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f84b7f2-611a-4c1c-9896-660edb3a1536 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:08.165725 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f84b7f2-611a-4c1c-9896-660edb3a1536 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:08.166216 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f84b7f2-611a-4c1c-9896-660edb3a1536 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:08.167611 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 691/1377] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:28:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:28:08.173517 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f10d818-65e6-4c06-adb4-85485fb36041 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:08.173983 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f10d818-65e6-4c06-adb4-85485fb36041 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:08.174089 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f10d818-65e6-4c06-adb4-85485fb36041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:08.239187 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-7f10d818-65e6-4c06-adb4-85485fb36041 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:28:08.240399 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-7f10d818-65e6-4c06-adb4-85485fb36041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:08.251448 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:28:08.356654 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 687/1378] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:28:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:28:08.409450 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f84b7f2-611a-4c1c-9896-660edb3a1536 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:28:08.410995 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-9f84b7f2-611a-4c1c-9896-660edb3a1536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:08.555963 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f84b7f2-611a-4c1c-9896-660edb3a1536 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:28:08.559567 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:28:08.671273 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f8884d-45d2-43a2-a038-e8cd15e4ee98 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7673a3b5c0324e07a9535ac70ed7a491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:28:08.671934 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f8884d-45d2-43a2-a038-e8cd15e4ee98 service nova] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:08.672214 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f8884d-45d2-43a2-a038-e8cd15e4ee98 service nova] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:08.672490 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f8884d-45d2-43a2-a038-e8cd15e4ee98 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:08.674199 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-52f8884d-45d2-43a2-a038-e8cd15e4ee98 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:08.796351 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f8884d-45d2-43a2-a038-e8cd15e4ee98 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:28:08.798443 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f8884d-45d2-43a2-a038-e8cd15e4ee98 service nova] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:28:08.830271 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 692/1379] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:28:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:28:11.024464 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b8b9b4-77c2-4935-99f6-de7416dda4eb None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:11.025023 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b8b9b4-77c2-4935-99f6-de7416dda4eb None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:11.025023 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b8b9b4-77c2-4935-99f6-de7416dda4eb None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:11.026661 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 688/1380] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:28:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:28:11.032564 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72fd941-8213-4603-822c-adead0b0d1d3 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:11.033018 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72fd941-8213-4603-822c-adead0b0d1d3 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:11.033335 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72fd941-8213-4603-822c-adead0b0d1d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:11.043818 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:28:11.106672 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-f72fd941-8213-4603-822c-adead0b0d1d3 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:28:11.107998 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-f72fd941-8213-4603-822c-adead0b0d1d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:11.215050 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 693/1381] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:28:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:28:11.260621 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b8b9b4-77c2-4935-99f6-de7416dda4eb None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:28:11.262269 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-16b8b9b4-77c2-4935-99f6-de7416dda4eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:11.400761 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:28:11.411573 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b8b9b4-77c2-4935-99f6-de7416dda4eb None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:28:11.542471 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e768ed2-4e4c-43da-9828-24bb5fd7b657 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7673a3b5c0324e07a9535ac70ed7a491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:28:11.543125 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e768ed2-4e4c-43da-9828-24bb5fd7b657 service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:11.543437 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e768ed2-4e4c-43da-9828-24bb5fd7b657 service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:11.543715 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e768ed2-4e4c-43da-9828-24bb5fd7b657 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:11.545370 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-5e768ed2-4e4c-43da-9828-24bb5fd7b657 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:11.643824 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e768ed2-4e4c-43da-9828-24bb5fd7b657 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:28:11.645352 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e768ed2-4e4c-43da-9828-24bb5fd7b657 service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:28:11.658504 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:28:11.666954 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 689/1382] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:28:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:28:14.126756 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f184732-34e2-4517-85f4-40f1ceefe8c4 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:14.127271 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f184732-34e2-4517-85f4-40f1ceefe8c4 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:14.127271 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f184732-34e2-4517-85f4-40f1ceefe8c4 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:14.128630 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 694/1383] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:28:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:28:14.134548 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c362df1-73c5-40de-8508-da4f890ad0c9 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:14.134977 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c362df1-73c5-40de-8508-da4f890ad0c9 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:14.135182 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c362df1-73c5-40de-8508-da4f890ad0c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:14.204662 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-5c362df1-73c5-40de-8508-da4f890ad0c9 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:28:14.205942 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-5c362df1-73c5-40de-8508-da4f890ad0c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:14.308824 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 690/1384] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:28:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:28:14.360099 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f184732-34e2-4517-85f4-40f1ceefe8c4 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:28:14.362257 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-7f184732-34e2-4517-85f4-40f1ceefe8c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:14.468134 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:28:14.538660 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f184732-34e2-4517-85f4-40f1ceefe8c4 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:28:14.675482 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81993889-bee6-48e8-bbe3-4de4f816a3ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7673a3b5c0324e07a9535ac70ed7a491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:28:14.676147 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81993889-bee6-48e8-bbe3-4de4f816a3ff service nova] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:14.676431 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81993889-bee6-48e8-bbe3-4de4f816a3ff service nova] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:14.676708 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81993889-bee6-48e8-bbe3-4de4f816a3ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:14.678492 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-81993889-bee6-48e8-bbe3-4de4f816a3ff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:14.739336 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:28:14.787087 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81993889-bee6-48e8-bbe3-4de4f816a3ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:28:14.788715 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81993889-bee6-48e8-bbe3-4de4f816a3ff service nova] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:28:14.807129 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 695/1385] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:28:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:28:16.900012 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee72eec-a4cc-497d-a4b9-1dcedafc32b1 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:16.900706 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee72eec-a4cc-497d-a4b9-1dcedafc32b1 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:16.900706 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee72eec-a4cc-497d-a4b9-1dcedafc32b1 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:16.902631 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 691/1386] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:28:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:28:16.909615 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fa2dd7-7a54-4240-8c52-52a7420d195e None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:16.910351 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fa2dd7-7a54-4240-8c52-52a7420d195e None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:16.910351 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fa2dd7-7a54-4240-8c52-52a7420d195e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:16.976483 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-30fa2dd7-7a54-4240-8c52-52a7420d195e None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:28:16.977724 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-30fa2dd7-7a54-4240-8c52-52a7420d195e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:17.086424 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 696/1387] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:28:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:28:17.141966 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee72eec-a4cc-497d-a4b9-1dcedafc32b1 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:28:17.144015 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-2ee72eec-a4cc-497d-a4b9-1dcedafc32b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:17.231329 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:28:17.305272 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee72eec-a4cc-497d-a4b9-1dcedafc32b1 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:28:17.470379 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c5cd49-5d30-406d-b540-857d0f1b08f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7673a3b5c0324e07a9535ac70ed7a491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:28:17.470948 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c5cd49-5d30-406d-b540-857d0f1b08f0 service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:17.471233 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c5cd49-5d30-406d-b540-857d0f1b08f0 service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:17.471511 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c5cd49-5d30-406d-b540-857d0f1b08f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:17.473181 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-32c5cd49-5d30-406d-b540-857d0f1b08f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:17.522246 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:28:17.575703 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c5cd49-5d30-406d-b540-857d0f1b08f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:28:17.577325 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c5cd49-5d30-406d-b540-857d0f1b08f0 service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:28:17.597087 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 692/1388] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:28:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:28:19.786201 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a321ba-d682-4064-82e7-71f65a4249ae None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:19.786708 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a321ba-d682-4064-82e7-71f65a4249ae None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:19.786708 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a321ba-d682-4064-82e7-71f65a4249ae None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:19.788080 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 697/1389] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:28:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:28:19.793230 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20928099-bc8e-461c-8f77-671809b7e870 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:19.793668 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20928099-bc8e-461c-8f77-671809b7e870 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:19.793828 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20928099-bc8e-461c-8f77-671809b7e870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:19.859604 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-20928099-bc8e-461c-8f77-671809b7e870 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:28:19.860842 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-20928099-bc8e-461c-8f77-671809b7e870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:19.968407 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 693/1390] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:28:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:28:20.013827 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a321ba-d682-4064-82e7-71f65a4249ae None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:28:20.015350 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-60a321ba-d682-4064-82e7-71f65a4249ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:20.045605 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:28:20.166063 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a321ba-d682-4064-82e7-71f65a4249ae None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:28:20.271026 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:28:20.282886 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcf93ee-6780-4f34-a052-d947dc2cc500 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7673a3b5c0324e07a9535ac70ed7a491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:28:20.283540 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcf93ee-6780-4f34-a052-d947dc2cc500 service nova] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:20.284537 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcf93ee-6780-4f34-a052-d947dc2cc500 service nova] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:20.284537 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcf93ee-6780-4f34-a052-d947dc2cc500 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:20.285802 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-9dcf93ee-6780-4f34-a052-d947dc2cc500 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:20.376762 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcf93ee-6780-4f34-a052-d947dc2cc500 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:28:20.378406 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcf93ee-6780-4f34-a052-d947dc2cc500 service nova] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:28:20.397502 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 698/1391] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:28:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:28:20.590651 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d988d84-facd-4b75-b307-a18a5ecc46fb None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:20.590935 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d988d84-facd-4b75-b307-a18a5ecc46fb None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:20.591211 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d988d84-facd-4b75-b307-a18a5ecc46fb None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:20.592714 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 694/1392] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:28:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:28:20.597902 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d0d6d7-ea51-43b3-b361-4335f2e69846 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:20.599626 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d0d6d7-ea51-43b3-b361-4335f2e69846 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:20.599626 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d0d6d7-ea51-43b3-b361-4335f2e69846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:20.673931 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-63d0d6d7-ea51-43b3-b361-4335f2e69846 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:28:20.675116 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-63d0d6d7-ea51-43b3-b361-4335f2e69846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:20.742821 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:28:20.786342 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 699/1393] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:28:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:28:20.843196 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c0ec03-9649-4eff-9e55-acdecbe5dfd4 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:20.843656 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c0ec03-9649-4eff-9e55-acdecbe5dfd4 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:20.843769 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c0ec03-9649-4eff-9e55-acdecbe5dfd4 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:20.845207 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 695/1394] 10.222.0.33 () {54 vars in 960 bytes} [Fri Jul 15 19:28:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:28:20.850422 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f0bab6-681e-48e0-868e-db9d88e2ea65 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:20.850842 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f0bab6-681e-48e0-868e-db9d88e2ea65 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:20.851128 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f0bab6-681e-48e0-868e-db9d88e2ea65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:20.928255 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-c6f0bab6-681e-48e0-868e-db9d88e2ea65 None None] MFA Rules not processed for user `04b5b06b0015450a95a28bbcc757cc8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:28:20.929391 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-c6f0bab6-681e-48e0-868e-db9d88e2ea65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:21.005693 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 700/1395] 10.222.0.33 () {58 vars in 1057 bytes} [Fri Jul 15 19:28:20 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:28:21.013323 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c0ec03-9649-4eff-9e55-acdecbe5dfd4 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:28:21.014992 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e5c0ec03-9649-4eff-9e55-acdecbe5dfd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:21.029319 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:28:21.120866 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c0ec03-9649-4eff-9e55-acdecbe5dfd4 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:28:21.197006 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0dae8c-bb75-45a9-8106-a0382cb3a679 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04b5b06b0015450a95a28bbcc757cc8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:28:21.197573 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0dae8c-bb75-45a9-8106-a0382cb3a679 service glance] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:21.197891 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0dae8c-bb75-45a9-8106-a0382cb3a679 service glance] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:21.198166 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0dae8c-bb75-45a9-8106-a0382cb3a679 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:21.200321 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-da0dae8c-bb75-45a9-8106-a0382cb3a679 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:21.267934 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:28:21.308889 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0dae8c-bb75-45a9-8106-a0382cb3a679 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:28:21.310755 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0dae8c-bb75-45a9-8106-a0382cb3a679 service glance] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:28:21.330208 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 696/1396] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:28:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:28:22.584750 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2921d8dc-ca86-438d-ae78-496916568208 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:22.585340 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2921d8dc-ca86-438d-ae78-496916568208 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:22.585443 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2921d8dc-ca86-438d-ae78-496916568208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:22.607488 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d07f707-935e-4d71-bc08-5874969251fc None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:22.608056 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d07f707-935e-4d71-bc08-5874969251fc None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:22.608056 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d07f707-935e-4d71-bc08-5874969251fc None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:22.609632 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 697/1397] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:28:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 19:28:22.614629 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1a0545-cab7-4cd4-8b21-3904af567a0b None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:22.614915 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1a0545-cab7-4cd4-8b21-3904af567a0b None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:22.615189 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1a0545-cab7-4cd4-8b21-3904af567a0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:22.655957 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-2921d8dc-ca86-438d-ae78-496916568208 None None] MFA Rules not processed for user `3d893c14620b4a129842062f14a68a78`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:28:22.657494 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-2921d8dc-ca86-438d-ae78-496916568208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:22.665862 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:28:22.682083 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-0f1a0545-cab7-4cd4-8b21-3904af567a0b None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:28:22.683303 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0f1a0545-cab7-4cd4-8b21-3904af567a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:22.746082 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 701/1398] 10.222.0.33 () {58 vars in 1009 bytes} [Fri Jul 15 19:28:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:28:22.792737 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 698/1399] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:28:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:28:22.796072 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c276fd83-3648-4d12-a587-7dbb3884b972 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:22.796353 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c276fd83-3648-4d12-a587-7dbb3884b972 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:22.796625 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c276fd83-3648-4d12-a587-7dbb3884b972 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:22.798035 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 702/1400] 10.222.0.33 () {54 vars in 942 bytes} [Fri Jul 15 19:28:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:28:22.807303 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7197ec21-edda-4f34-a0fb-42f404745f5a None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:22.807628 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7197ec21-edda-4f34-a0fb-42f404745f5a None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:22.807876 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7197ec21-edda-4f34-a0fb-42f404745f5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:22.833745 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:28:22.835234 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c276fd83-3648-4d12-a587-7dbb3884b972 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:28:22.836781 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-c276fd83-3648-4d12-a587-7dbb3884b972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:22.884835 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-7197ec21-edda-4f34-a0fb-42f404745f5a None None] MFA Rules not processed for user `ef76f7fe39a3499aaeebf85c176804f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:28:22.886161 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-7197ec21-edda-4f34-a0fb-42f404745f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:22.988453 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c276fd83-3648-4d12-a587-7dbb3884b972 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:28:22.992363 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:28:23.004937 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 699/1401] 10.222.0.33 () {58 vars in 1039 bytes} [Fri Jul 15 19:28:22 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:28:23.015231 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7197ec21-edda-4f34-a0fb-42f404745f5a None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:28:23.025220 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7197ec21-edda-4f34-a0fb-42f404745f5a None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:28:23.099250 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:28:23.122740 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cf6705-be6f-4604-ae17-b810cf5c9a07 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7673a3b5c0324e07a9535ac70ed7a491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:28:23.123389 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cf6705-be6f-4604-ae17-b810cf5c9a07 service nova] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:23.123667 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cf6705-be6f-4604-ae17-b810cf5c9a07 service nova] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:23.123936 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cf6705-be6f-4604-ae17-b810cf5c9a07 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:23.125662 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-21cf6705-be6f-4604-ae17-b810cf5c9a07 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:23.138628 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7472feef-df65-43d2-8f42-dd8ef3fb8c85 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef76f7fe39a3499aaeebf85c176804f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:28:23.139411 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7472feef-df65-43d2-8f42-dd8ef3fb8c85 service swift] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:23.140555 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7472feef-df65-43d2-8f42-dd8ef3fb8c85 service swift] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:23.141126 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7472feef-df65-43d2-8f42-dd8ef3fb8c85 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:23.143689 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-7472feef-df65-43d2-8f42-dd8ef3fb8c85 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:23.221705 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cf6705-be6f-4604-ae17-b810cf5c9a07 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:28:23.223327 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cf6705-be6f-4604-ae17-b810cf5c9a07 service nova] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:28:23.230205 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7472feef-df65-43d2-8f42-dd8ef3fb8c85 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:28:23.231818 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7472feef-df65-43d2-8f42-dd8ef3fb8c85 service swift] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:28:23.240549 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 700/1402] 10.222.0.33 () {60 vars in 1384 bytes} [Fri Jul 15 19:28:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 15 19:28:23.241198 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 703/1403] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:28:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 19:28:25.459599 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff16c986-dc6a-46ce-b464-82eba6870058 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:25.460195 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff16c986-dc6a-46ce-b464-82eba6870058 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:25.460195 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff16c986-dc6a-46ce-b464-82eba6870058 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:25.461628 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 701/1404] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:28:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:28:25.468035 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9ad9d2-a256-46f0-b6de-a6c18e328106 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:25.468613 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9ad9d2-a256-46f0-b6de-a6c18e328106 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:25.468613 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9ad9d2-a256-46f0-b6de-a6c18e328106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:25.480429 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:28:25.561735 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-2c9ad9d2-a256-46f0-b6de-a6c18e328106 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:28:25.564740 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-2c9ad9d2-a256-46f0-b6de-a6c18e328106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:25.689938 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 704/1405] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:28:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:28:25.733793 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff16c986-dc6a-46ce-b464-82eba6870058 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:28:25.735485 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-ff16c986-dc6a-46ce-b464-82eba6870058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:25.866394 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:28:25.917931 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff16c986-dc6a-46ce-b464-82eba6870058 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:28:26.066418 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7a9071-8c18-40e1-88f0-ed5d5c9620d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7673a3b5c0324e07a9535ac70ed7a491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:28:26.067599 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7a9071-8c18-40e1-88f0-ed5d5c9620d1 service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:26.068060 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7a9071-8c18-40e1-88f0-ed5d5c9620d1 service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:26.068340 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7a9071-8c18-40e1-88f0-ed5d5c9620d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:26.070069 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-2f7a9071-8c18-40e1-88f0-ed5d5c9620d1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:26.168493 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7a9071-8c18-40e1-88f0-ed5d5c9620d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:28:26.170196 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7a9071-8c18-40e1-88f0-ed5d5c9620d1 service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:28:26.192868 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 702/1406] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:28:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:28:26.555918 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:28:28.709115 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611d57a6-e765-4356-9e00-b55cf01051b0 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:28.710181 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611d57a6-e765-4356-9e00-b55cf01051b0 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:28.710181 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611d57a6-e765-4356-9e00-b55cf01051b0 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:28.711065 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 705/1407] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:28:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:28:28.716394 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d558a-e29c-47ec-80e6-6edf51702ac9 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:28.716818 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d558a-e29c-47ec-80e6-6edf51702ac9 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:28.716936 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d558a-e29c-47ec-80e6-6edf51702ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:28.792380 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-616d558a-e29c-47ec-80e6-6edf51702ac9 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:28:28.793597 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-616d558a-e29c-47ec-80e6-6edf51702ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:28.907225 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 703/1408] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:28:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:28:28.953101 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611d57a6-e765-4356-9e00-b55cf01051b0 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:28:28.954790 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-611d57a6-e765-4356-9e00-b55cf01051b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:29.046319 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:28:29.118506 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611d57a6-e765-4356-9e00-b55cf01051b0 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:28:29.241493 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439fff24-acbb-4999-8b1c-e4bfe6d54676 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7673a3b5c0324e07a9535ac70ed7a491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:28:29.242172 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439fff24-acbb-4999-8b1c-e4bfe6d54676 service nova] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:29.242453 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439fff24-acbb-4999-8b1c-e4bfe6d54676 service nova] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:29.242728 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439fff24-acbb-4999-8b1c-e4bfe6d54676 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:29.244394 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-439fff24-acbb-4999-8b1c-e4bfe6d54676 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:29.298862 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:28:29.342636 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439fff24-acbb-4999-8b1c-e4bfe6d54676 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:28:29.344228 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439fff24-acbb-4999-8b1c-e4bfe6d54676 service nova] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:28:29.366506 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 706/1409] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:28:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:28:38.905421 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=79 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:28:39.364210 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=23 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:28:47.154631 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216e84b3-0bfc-41fd-90b4-13381643f338 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:47.155200 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216e84b3-0bfc-41fd-90b4-13381643f338 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:47.155200 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216e84b3-0bfc-41fd-90b4-13381643f338 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:47.156617 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 704/1410] 10.222.0.33 () {54 vars in 918 bytes} [Fri Jul 15 19:28:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:28:47.161874 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807dde72-78af-4806-8ffd-a4bc0ca3d289 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:47.162380 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807dde72-78af-4806-8ffd-a4bc0ca3d289 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:47.162595 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807dde72-78af-4806-8ffd-a4bc0ca3d289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:47.235716 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-807dde72-78af-4806-8ffd-a4bc0ca3d289 None None] MFA Rules not processed for user `52a0fce231ba4785a685ac9898d07503`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:28:47.236933 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-807dde72-78af-4806-8ffd-a4bc0ca3d289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:47.342183 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 707/1411] 10.222.0.33 () {58 vars in 1015 bytes} [Fri Jul 15 19:28:47 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:28:47.356435 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d0d7b4-0492-46c8-bfaa-adf3275af862 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:47.356742 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d0d7b4-0492-46c8-bfaa-adf3275af862 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:47.357017 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d0d7b4-0492-46c8-bfaa-adf3275af862 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:47.358456 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 705/1412] 10.222.0.33 () {54 vars in 918 bytes} [Fri Jul 15 19:28:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:28:47.363159 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229c18b5-05e0-4588-af55-11496246364a None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:47.363441 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229c18b5-05e0-4588-af55-11496246364a None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:47.363720 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229c18b5-05e0-4588-af55-11496246364a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:47.435429 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:28:47.438061 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-229c18b5-05e0-4588-af55-11496246364a None None] MFA Rules not processed for user `7673a3b5c0324e07a9535ac70ed7a491`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:28:47.439288 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-229c18b5-05e0-4588-af55-11496246364a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:47.555917 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 708/1413] 10.222.0.33 () {58 vars in 1015 bytes} [Fri Jul 15 19:28:47 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:28:50.015338 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45816e52-9ff4-47e2-b870-1d382fb1aa9e None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:50.015879 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45816e52-9ff4-47e2-b870-1d382fb1aa9e None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:50.015879 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45816e52-9ff4-47e2-b870-1d382fb1aa9e None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:50.017262 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 706/1414] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 19:28:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:28:50.022973 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c5c719-7ab8-4290-99cc-bd4bb42c4b7d None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:50.023380 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c5c719-7ab8-4290-99cc-bd4bb42c4b7d None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:50.023531 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c5c719-7ab8-4290-99cc-bd4bb42c4b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:50.104467 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-23c5c719-7ab8-4290-99cc-bd4bb42c4b7d None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:28:50.105733 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-23c5c719-7ab8-4290-99cc-bd4bb42c4b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:50.161293 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:28:50.205495 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 709/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 19:28:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:28:50.247792 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45816e52-9ff4-47e2-b870-1d382fb1aa9e None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:28:50.249318 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-45816e52-9ff4-47e2-b870-1d382fb1aa9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:50.393645 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45816e52-9ff4-47e2-b870-1d382fb1aa9e None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:28:50.467850 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:28:50.509273 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1774184-0c94-45f8-b706-b5fad2b0b165 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7673a3b5c0324e07a9535ac70ed7a491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:28:50.509955 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1774184-0c94-45f8-b706-b5fad2b0b165 service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:50.510239 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1774184-0c94-45f8-b706-b5fad2b0b165 service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:50.510517 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1774184-0c94-45f8-b706-b5fad2b0b165 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:50.512758 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-c1774184-0c94-45f8-b706-b5fad2b0b165 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:50.601685 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1774184-0c94-45f8-b706-b5fad2b0b165 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:28:50.603373 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1774184-0c94-45f8-b706-b5fad2b0b165 service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:28:50.621414 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 707/1416] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:28:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:28:51.513322 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aa9ca3-6e4e-4be1-be42-a739df415d6a None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:51.513882 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aa9ca3-6e4e-4be1-be42-a739df415d6a None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:51.513882 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aa9ca3-6e4e-4be1-be42-a739df415d6a None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:51.515229 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 710/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 15 19:28:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:28:51.520098 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822db1b2-9b02-466a-9543-0c24b7909244 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:51.520529 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822db1b2-9b02-466a-9543-0c24b7909244 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:51.520633 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822db1b2-9b02-466a-9543-0c24b7909244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:51.596686 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-822db1b2-9b02-466a-9543-0c24b7909244 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:28:51.597930 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-822db1b2-9b02-466a-9543-0c24b7909244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:51.624285 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:28:51.694032 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 708/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 15 19:28:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:28:51.913290 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aa9ca3-6e4e-4be1-be42-a739df415d6a None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:28:51.913290 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-74aa9ca3-6e4e-4be1-be42-a739df415d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:51.997372 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aa9ca3-6e4e-4be1-be42-a739df415d6a None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:28:52.038613 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:28:52.110286 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480c202e-1bca-4854-ba59-6fc1a9370179 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7673a3b5c0324e07a9535ac70ed7a491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:28:52.110857 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480c202e-1bca-4854-ba59-6fc1a9370179 service nova] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:52.111141 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480c202e-1bca-4854-ba59-6fc1a9370179 service nova] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:52.111417 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480c202e-1bca-4854-ba59-6fc1a9370179 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:52.113065 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-480c202e-1bca-4854-ba59-6fc1a9370179 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:52.202002 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480c202e-1bca-4854-ba59-6fc1a9370179 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:28:52.203599 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480c202e-1bca-4854-ba59-6fc1a9370179 service nova] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:28:52.221891 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 711/1419] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:28:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:28:53.100250 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af26a9ba-2d81-475d-90e8-87ec86710527 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:53.100683 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af26a9ba-2d81-475d-90e8-87ec86710527 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:53.100823 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af26a9ba-2d81-475d-90e8-87ec86710527 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:53.102266 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 709/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 15 19:28:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:28:53.107226 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01211b53-5896-440a-978c-9786a9c1f07f None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:53.107630 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01211b53-5896-440a-978c-9786a9c1f07f None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:53.107770 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01211b53-5896-440a-978c-9786a9c1f07f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:53.162124 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:28:53.183462 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-01211b53-5896-440a-978c-9786a9c1f07f None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:28:53.184661 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-01211b53-5896-440a-978c-9786a9c1f07f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:53.279899 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 712/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 15 19:28:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:28:53.430431 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af26a9ba-2d81-475d-90e8-87ec86710527 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:28:53.431903 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-af26a9ba-2d81-475d-90e8-87ec86710527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:53.579635 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af26a9ba-2d81-475d-90e8-87ec86710527 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:28:53.596333 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:28:53.702312 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2002ce07-2648-4916-a3dd-58de226c644d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7673a3b5c0324e07a9535ac70ed7a491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:28:53.702874 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2002ce07-2648-4916-a3dd-58de226c644d service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:53.703153 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2002ce07-2648-4916-a3dd-58de226c644d service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:53.703423 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2002ce07-2648-4916-a3dd-58de226c644d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:53.705072 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-2002ce07-2648-4916-a3dd-58de226c644d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:53.794409 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2002ce07-2648-4916-a3dd-58de226c644d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:28:53.795998 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2002ce07-2648-4916-a3dd-58de226c644d service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:28:53.813937 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 710/1422] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:28:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:28:54.667797 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414fc5b9-940f-4c31-a5a4-3fa76b7a1061 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:54.668424 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414fc5b9-940f-4c31-a5a4-3fa76b7a1061 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:54.668424 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414fc5b9-940f-4c31-a5a4-3fa76b7a1061 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:54.669721 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 713/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 15 19:28:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:28:54.674638 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84c8253-7091-4a3a-ba73-a88da645455d None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:54.675075 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84c8253-7091-4a3a-ba73-a88da645455d None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:54.675227 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84c8253-7091-4a3a-ba73-a88da645455d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:54.693401 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:28:54.749100 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-b84c8253-7091-4a3a-ba73-a88da645455d None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:28:54.750322 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b84c8253-7091-4a3a-ba73-a88da645455d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:54.848274 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 711/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 15 19:28:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:28:54.997611 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414fc5b9-940f-4c31-a5a4-3fa76b7a1061 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:28:54.999105 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-414fc5b9-940f-4c31-a5a4-3fa76b7a1061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:55.106000 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:28:55.140539 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414fc5b9-940f-4c31-a5a4-3fa76b7a1061 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:28:55.251110 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d673617-1027-459e-8ad7-3dc4528f26ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7673a3b5c0324e07a9535ac70ed7a491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:28:55.251674 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d673617-1027-459e-8ad7-3dc4528f26ba service nova] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:55.251951 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d673617-1027-459e-8ad7-3dc4528f26ba service nova] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:55.252221 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d673617-1027-459e-8ad7-3dc4528f26ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:55.253883 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-0d673617-1027-459e-8ad7-3dc4528f26ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:55.328154 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:28:55.341193 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d673617-1027-459e-8ad7-3dc4528f26ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:28:55.342811 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d673617-1027-459e-8ad7-3dc4528f26ba service nova] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:28:55.360236 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 714/1425] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:28:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:28:56.231152 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69193d63-e6a0-4d22-8121-78a4b592b810 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:56.231690 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69193d63-e6a0-4d22-8121-78a4b592b810 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:56.231690 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69193d63-e6a0-4d22-8121-78a4b592b810 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:56.233134 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 712/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 15 19:28:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:28:56.238193 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae30bc29-5ed7-4882-a6ba-c074575dc23a None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:56.238625 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae30bc29-5ed7-4882-a6ba-c074575dc23a None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:56.238753 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae30bc29-5ed7-4882-a6ba-c074575dc23a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:56.430523 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-ae30bc29-5ed7-4882-a6ba-c074575dc23a None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:28:56.431692 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-ae30bc29-5ed7-4882-a6ba-c074575dc23a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:56.527299 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 715/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 15 19:28:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:28:56.675718 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69193d63-e6a0-4d22-8121-78a4b592b810 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:28:56.677241 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-69193d63-e6a0-4d22-8121-78a4b592b810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:56.761492 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:28:56.824823 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69193d63-e6a0-4d22-8121-78a4b592b810 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:28:56.945911 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33806fc7-87e7-4736-9d00-a8ae7d2b1475 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7673a3b5c0324e07a9535ac70ed7a491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:28:56.946562 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33806fc7-87e7-4736-9d00-a8ae7d2b1475 service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:56.946846 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33806fc7-87e7-4736-9d00-a8ae7d2b1475 service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:56.947122 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33806fc7-87e7-4736-9d00-a8ae7d2b1475 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:56.949396 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-33806fc7-87e7-4736-9d00-a8ae7d2b1475 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:56.996335 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:28:57.037507 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33806fc7-87e7-4736-9d00-a8ae7d2b1475 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:28:57.039092 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33806fc7-87e7-4736-9d00-a8ae7d2b1475 service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:28:57.056968 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 713/1428] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:28:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:28:57.906428 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3152c8f0-e3da-4c73-b9f6-278b8bdae75a None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:57.906894 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3152c8f0-e3da-4c73-b9f6-278b8bdae75a None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:57.907016 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3152c8f0-e3da-4c73-b9f6-278b8bdae75a None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:57.908352 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 716/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 15 19:28:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:28:57.913477 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1999ae79-00c8-49dc-bd1d-f1a55133d76f None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:57.913926 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1999ae79-00c8-49dc-bd1d-f1a55133d76f None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:57.914064 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1999ae79-00c8-49dc-bd1d-f1a55133d76f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:57.989847 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-1999ae79-00c8-49dc-bd1d-f1a55133d76f None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:28:57.990972 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-1999ae79-00c8-49dc-bd1d-f1a55133d76f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:58.085252 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 714/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 15 19:28:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:28:58.240716 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3152c8f0-e3da-4c73-b9f6-278b8bdae75a None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:28:58.242243 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-3152c8f0-e3da-4c73-b9f6-278b8bdae75a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:58.291019 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:28:58.385680 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3152c8f0-e3da-4c73-b9f6-278b8bdae75a None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:28:58.505279 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fd0565-2b40-4d1e-be7b-9d5cc21f98a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7673a3b5c0324e07a9535ac70ed7a491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:28:58.505888 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd0565-2b40-4d1e-be7b-9d5cc21f98a1 service nova] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:58.506242 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd0565-2b40-4d1e-be7b-9d5cc21f98a1 service nova] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:58.506510 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd0565-2b40-4d1e-be7b-9d5cc21f98a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:58.508157 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-e7fd0565-2b40-4d1e-be7b-9d5cc21f98a1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:58.515556 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:28:58.603051 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fd0565-2b40-4d1e-be7b-9d5cc21f98a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:28:58.605325 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fd0565-2b40-4d1e-be7b-9d5cc21f98a1 service nova] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:28:58.623251 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 717/1431] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:28:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:28:59.490280 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03946821-bd10-4658-9db8-fc554e42568b None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:59.496505 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03946821-bd10-4658-9db8-fc554e42568b None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:59.496505 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03946821-bd10-4658-9db8-fc554e42568b None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:59.496505 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 715/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 15 19:28:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:28:59.497737 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a369c290-527e-4c51-bf7f-2e85404a83d3 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:59.498165 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a369c290-527e-4c51-bf7f-2e85404a83d3 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:59.498289 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a369c290-527e-4c51-bf7f-2e85404a83d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:59.574256 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-a369c290-527e-4c51-bf7f-2e85404a83d3 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:28:59.575477 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-a369c290-527e-4c51-bf7f-2e85404a83d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:59.642179 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:28:59.671362 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 718/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 15 19:28:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:28:59.822200 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03946821-bd10-4658-9db8-fc554e42568b None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:28:59.823769 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-03946821-bd10-4658-9db8-fc554e42568b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:59.841596 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:28:59.969862 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03946821-bd10-4658-9db8-fc554e42568b None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:29:00.061786 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:29:00.086728 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68f942a-67a3-46f4-bfd4-c836e881c1e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7673a3b5c0324e07a9535ac70ed7a491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:29:00.087381 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f942a-67a3-46f4-bfd4-c836e881c1e0 service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:29:00.087664 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f942a-67a3-46f4-bfd4-c836e881c1e0 service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:29:00.087941 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f942a-67a3-46f4-bfd4-c836e881c1e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:29:00.090136 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d68f942a-67a3-46f4-bfd4-c836e881c1e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:29:00.180442 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68f942a-67a3-46f4-bfd4-c836e881c1e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:29:00.182086 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68f942a-67a3-46f4-bfd4-c836e881c1e0 service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:29:00.200507 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 716/1434] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:28:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:29:01.067493 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8b9dd1-5571-422e-9f6b-3dab5194711c None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:29:01.068135 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8b9dd1-5571-422e-9f6b-3dab5194711c None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:29:01.068135 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8b9dd1-5571-422e-9f6b-3dab5194711c None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:29:01.069317 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 719/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 15 19:29:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:29:01.074720 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de5822c-ad3d-4cd1-bef8-798d97be91ca None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:29:01.075155 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de5822c-ad3d-4cd1-bef8-798d97be91ca None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:29:01.075286 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de5822c-ad3d-4cd1-bef8-798d97be91ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:29:01.150705 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-3de5822c-ad3d-4cd1-bef8-798d97be91ca None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:29:01.152560 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-3de5822c-ad3d-4cd1-bef8-798d97be91ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:29:01.205698 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:29:01.267324 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 717/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 15 19:29:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:29:01.415729 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8b9dd1-5571-422e-9f6b-3dab5194711c None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:29:01.417850 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-ad8b9dd1-5571-422e-9f6b-3dab5194711c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:29:01.568734 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8b9dd1-5571-422e-9f6b-3dab5194711c None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:29:01.621862 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:29:01.680625 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfccfb4-59f9-4693-9519-fefc8a4a47ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7673a3b5c0324e07a9535ac70ed7a491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:29:01.681205 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfccfb4-59f9-4693-9519-fefc8a4a47ec service nova] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:29:01.681487 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfccfb4-59f9-4693-9519-fefc8a4a47ec service nova] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:29:01.681758 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfccfb4-59f9-4693-9519-fefc8a4a47ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:29:01.684013 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-3cfccfb4-59f9-4693-9519-fefc8a4a47ec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:29:01.772682 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfccfb4-59f9-4693-9519-fefc8a4a47ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:29:01.774317 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfccfb4-59f9-4693-9519-fefc8a4a47ec service nova] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:29:01.792981 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 720/1437] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:29:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:29:02.685543 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8d11fd-4d5a-4ee2-9b63-3d9cd2422d76 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:29:02.686120 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8d11fd-4d5a-4ee2-9b63-3d9cd2422d76 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:29:02.686120 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8d11fd-4d5a-4ee2-9b63-3d9cd2422d76 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:29:02.687518 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 718/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 15 19:29:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:29:02.692444 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f0c9eb-c305-4630-8e0e-b87f061f7f99 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:29:02.692926 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f0c9eb-c305-4630-8e0e-b87f061f7f99 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:29:02.693030 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f0c9eb-c305-4630-8e0e-b87f061f7f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:29:02.769112 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-41f0c9eb-c305-4630-8e0e-b87f061f7f99 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:29:02.770343 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-41f0c9eb-c305-4630-8e0e-b87f061f7f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:29:02.772611 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:29:02.867061 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 721/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 15 19:29:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:29:03.018883 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8d11fd-4d5a-4ee2-9b63-3d9cd2422d76 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:29:03.020893 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-bf8d11fd-4d5a-4ee2-9b63-3d9cd2422d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:29:03.165412 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8d11fd-4d5a-4ee2-9b63-3d9cd2422d76 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:29:03.193179 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:29:03.278477 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895ce311-bc7c-4257-a117-730ed9b985c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7673a3b5c0324e07a9535ac70ed7a491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:29:03.279048 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895ce311-bc7c-4257-a117-730ed9b985c9 service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:29:03.279324 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895ce311-bc7c-4257-a117-730ed9b985c9 service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:29:03.279596 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895ce311-bc7c-4257-a117-730ed9b985c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:29:03.281234 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-895ce311-bc7c-4257-a117-730ed9b985c9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:29:03.369063 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895ce311-bc7c-4257-a117-730ed9b985c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:29:03.371275 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895ce311-bc7c-4257-a117-730ed9b985c9 service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:29:03.389689 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 719/1440] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:29:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:29:05.319879 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dacf77b-cbf9-48d2-b1e6-7afe30378b42 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:29:05.320437 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dacf77b-cbf9-48d2-b1e6-7afe30378b42 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:29:05.320437 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dacf77b-cbf9-48d2-b1e6-7afe30378b42 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:29:05.321840 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 722/1441] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 19:29:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:29:05.327556 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a48923-5e77-40d2-af32-a47391921fee None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:29:05.328010 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a48923-5e77-40d2-af32-a47391921fee None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:29:05.328120 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a48923-5e77-40d2-af32-a47391921fee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:29:05.366618 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:29:05.391678 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-72a48923-5e77-40d2-af32-a47391921fee None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:29:05.392906 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-72a48923-5e77-40d2-af32-a47391921fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:29:05.489425 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 720/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 19:29:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:29:05.529440 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dacf77b-cbf9-48d2-b1e6-7afe30378b42 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:29:05.531121 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-7dacf77b-cbf9-48d2-b1e6-7afe30378b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:29:05.648870 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dacf77b-cbf9-48d2-b1e6-7afe30378b42 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:29:05.680936 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:29:05.721708 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d5e2c2-8841-458d-8cc7-1f5843bd1167 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:29:05.722399 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d5e2c2-8841-458d-8cc7-1f5843bd1167 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:29:05.722667 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d5e2c2-8841-458d-8cc7-1f5843bd1167 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:29:05.722939 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d5e2c2-8841-458d-8cc7-1f5843bd1167 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:29:05.724585 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-61d5e2c2-8841-458d-8cc7-1f5843bd1167 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:29:05.833558 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d5e2c2-8841-458d-8cc7-1f5843bd1167 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:29:05.835035 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d5e2c2-8841-458d-8cc7-1f5843bd1167 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:29:05.854902 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 723/1443] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:29:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:29:08.016467 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9aa81d-005b-42f2-b636-b7a27232d696 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:29:08.016987 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9aa81d-005b-42f2-b636-b7a27232d696 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:29:08.016987 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9aa81d-005b-42f2-b636-b7a27232d696 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:29:08.018367 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 721/1444] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 19:29:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:29:08.023945 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180859a-de30-4342-a430-7c1f058f44dd None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:29:08.024350 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180859a-de30-4342-a430-7c1f058f44dd None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:29:08.024504 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180859a-de30-4342-a430-7c1f058f44dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:29:08.093502 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-3180859a-de30-4342-a430-7c1f058f44dd None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:29:08.095385 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-3180859a-de30-4342-a430-7c1f058f44dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:29:08.108716 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:29:08.192587 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 724/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 19:29:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:29:08.236001 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9aa81d-005b-42f2-b636-b7a27232d696 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:29:08.237584 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-2e9aa81d-005b-42f2-b636-b7a27232d696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:29:08.341524 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9aa81d-005b-42f2-b636-b7a27232d696 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:29:08.353913 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:29:08.416288 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e721e4-193b-4222-a8b6-4addaa540423 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:29:08.417429 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e721e4-193b-4222-a8b6-4addaa540423 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:29:08.417722 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e721e4-193b-4222-a8b6-4addaa540423 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:29:08.418026 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e721e4-193b-4222-a8b6-4addaa540423 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:29:08.419696 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-83e721e4-193b-4222-a8b6-4addaa540423 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:29:08.529443 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e721e4-193b-4222-a8b6-4addaa540423 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:29:08.530938 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e721e4-193b-4222-a8b6-4addaa540423 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:29:08.549096 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 722/1446] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:29:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:29:10.929837 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caec209-4265-491c-96a4-bd9291cd8317 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:29:10.930393 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caec209-4265-491c-96a4-bd9291cd8317 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:29:10.930393 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caec209-4265-491c-96a4-bd9291cd8317 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:29:10.931804 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 725/1447] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 19:29:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:29:10.937703 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e7ec60-516f-4170-bb0a-437561d6d102 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:29:10.938223 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e7ec60-516f-4170-bb0a-437561d6d102 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:29:10.938331 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e7ec60-516f-4170-bb0a-437561d6d102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:29:10.984011 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:29:11.001889 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-a0e7ec60-516f-4170-bb0a-437561d6d102 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:29:11.003159 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-a0e7ec60-516f-4170-bb0a-437561d6d102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:29:11.098819 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 723/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 19:29:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:29:11.138028 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caec209-4265-491c-96a4-bd9291cd8317 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:29:11.139599 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-8caec209-4265-491c-96a4-bd9291cd8317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:29:11.241922 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caec209-4265-491c-96a4-bd9291cd8317 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:29:11.281748 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:29:11.315621 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7797ab4-9cd2-40e6-8d3d-dc165d727c88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:29:11.316267 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7797ab4-9cd2-40e6-8d3d-dc165d727c88 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:29:11.316550 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7797ab4-9cd2-40e6-8d3d-dc165d727c88 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:29:11.316845 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7797ab4-9cd2-40e6-8d3d-dc165d727c88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:29:11.318523 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-e7797ab4-9cd2-40e6-8d3d-dc165d727c88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:29:11.427170 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7797ab4-9cd2-40e6-8d3d-dc165d727c88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:29:11.428581 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7797ab4-9cd2-40e6-8d3d-dc165d727c88 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:29:11.447600 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 726/1449] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:29:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:29:13.638597 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8978c23-3077-462d-96a5-07276205de6b None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:29:13.639165 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8978c23-3077-462d-96a5-07276205de6b None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:29:13.639165 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8978c23-3077-462d-96a5-07276205de6b None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:29:13.640558 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 724/1450] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 19:29:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:29:13.646183 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef31c099-71d3-4b40-af54-144b1bbf07e1 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:29:13.646622 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef31c099-71d3-4b40-af54-144b1bbf07e1 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:29:13.646744 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef31c099-71d3-4b40-af54-144b1bbf07e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:29:13.710714 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-ef31c099-71d3-4b40-af54-144b1bbf07e1 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:29:13.711924 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-ef31c099-71d3-4b40-af54-144b1bbf07e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:29:13.732307 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:29:13.808710 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 727/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 19:29:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:29:13.846474 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8978c23-3077-462d-96a5-07276205de6b None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:29:13.848080 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-f8978c23-3077-462d-96a5-07276205de6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:29:13.960706 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8978c23-3077-462d-96a5-07276205de6b None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:29:13.981178 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:29:14.035064 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f487d52-cd36-45bb-8adc-ec5a84d4eb2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:29:14.035730 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f487d52-cd36-45bb-8adc-ec5a84d4eb2a service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:29:14.036017 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f487d52-cd36-45bb-8adc-ec5a84d4eb2a service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:29:14.036292 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f487d52-cd36-45bb-8adc-ec5a84d4eb2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:29:14.038000 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-5f487d52-cd36-45bb-8adc-ec5a84d4eb2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:29:14.157408 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f487d52-cd36-45bb-8adc-ec5a84d4eb2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:29:14.158895 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f487d52-cd36-45bb-8adc-ec5a84d4eb2a service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:29:14.176895 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 725/1452] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:29:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:29:16.235540 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2386f7-ff9d-4cd5-9072-fac29b6995f3 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:29:16.236081 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2386f7-ff9d-4cd5-9072-fac29b6995f3 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:29:16.236081 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2386f7-ff9d-4cd5-9072-fac29b6995f3 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:29:16.237381 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 728/1453] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 19:29:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:29:16.242903 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcf40e0-d1e9-47c3-8f4c-5e46840aa39f None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:29:16.243323 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcf40e0-d1e9-47c3-8f4c-5e46840aa39f None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:29:16.243440 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcf40e0-d1e9-47c3-8f4c-5e46840aa39f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:29:16.313898 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:29:16.317647 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-0dcf40e0-d1e9-47c3-8f4c-5e46840aa39f None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:29:16.319489 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0dcf40e0-d1e9-47c3-8f4c-5e46840aa39f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:29:16.422875 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 726/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 19:29:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:29:16.461505 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2386f7-ff9d-4cd5-9072-fac29b6995f3 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:29:16.463137 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-ef2386f7-ff9d-4cd5-9072-fac29b6995f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:29:16.566908 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2386f7-ff9d-4cd5-9072-fac29b6995f3 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:29:16.615643 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:29:16.647556 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744e3737-c904-4cb5-8e5d-a66f90acfb3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:29:16.648404 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744e3737-c904-4cb5-8e5d-a66f90acfb3c service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:29:16.648771 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744e3737-c904-4cb5-8e5d-a66f90acfb3c service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:29:16.649159 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744e3737-c904-4cb5-8e5d-a66f90acfb3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:29:16.651776 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-744e3737-c904-4cb5-8e5d-a66f90acfb3c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:29:16.763681 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744e3737-c904-4cb5-8e5d-a66f90acfb3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:29:16.765102 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744e3737-c904-4cb5-8e5d-a66f90acfb3c service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:29:16.784541 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 729/1455] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:29:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:29:18.839215 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30a9de-1c43-4788-92f5-6ab7597c494e None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:29:18.839769 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30a9de-1c43-4788-92f5-6ab7597c494e None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:29:18.839769 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30a9de-1c43-4788-92f5-6ab7597c494e None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:29:18.841180 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 727/1456] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 19:29:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:29:18.846644 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c93b1c-d1df-4201-bbc1-c05b2a63d499 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:29:18.847086 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c93b1c-d1df-4201-bbc1-c05b2a63d499 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:29:18.850449 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c93b1c-d1df-4201-bbc1-c05b2a63d499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:29:18.911285 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-10c93b1c-d1df-4201-bbc1-c05b2a63d499 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:29:18.912481 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-10c93b1c-d1df-4201-bbc1-c05b2a63d499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:29:18.942821 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:29:19.009391 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 730/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 19:29:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:29:19.048577 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b30a9de-1c43-4788-92f5-6ab7597c494e None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:29:19.050216 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-6b30a9de-1c43-4788-92f5-6ab7597c494e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:29:19.152422 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b30a9de-1c43-4788-92f5-6ab7597c494e None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:29:19.179987 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:29:19.225943 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9b2557-d4f6-4412-abe3-3d876e763c38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:29:19.226513 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9b2557-d4f6-4412-abe3-3d876e763c38 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:29:19.226789 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9b2557-d4f6-4412-abe3-3d876e763c38 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:29:19.227059 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9b2557-d4f6-4412-abe3-3d876e763c38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:29:19.228724 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-2e9b2557-d4f6-4412-abe3-3d876e763c38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:29:19.336485 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9b2557-d4f6-4412-abe3-3d876e763c38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:29:19.337949 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9b2557-d4f6-4412-abe3-3d876e763c38 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:29:19.355533 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 728/1458] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:29:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:29:21.426748 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fd0f0f-f3c0-4ce9-901b-2c12320fe2c5 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:29:21.427281 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fd0f0f-f3c0-4ce9-901b-2c12320fe2c5 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:29:21.427281 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fd0f0f-f3c0-4ce9-901b-2c12320fe2c5 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:29:21.428615 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 731/1459] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 19:29:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:29:21.434601 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8874c6-65b0-4beb-9609-f25961422d88 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:29:21.435004 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8874c6-65b0-4beb-9609-f25961422d88 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:29:21.435190 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8874c6-65b0-4beb-9609-f25961422d88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:29:21.500359 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-cf8874c6-65b0-4beb-9609-f25961422d88 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:29:21.501592 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-cf8874c6-65b0-4beb-9609-f25961422d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:29:21.509935 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:29:21.598265 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 729/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 19:29:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:29:21.636788 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fd0f0f-f3c0-4ce9-901b-2c12320fe2c5 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:29:21.638491 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-a2fd0f0f-f3c0-4ce9-901b-2c12320fe2c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:29:21.740634 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fd0f0f-f3c0-4ce9-901b-2c12320fe2c5 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:29:21.798076 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:29:21.816290 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19a5127-b74a-42e1-b70d-b9773250051e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:29:21.817302 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19a5127-b74a-42e1-b70d-b9773250051e service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:29:21.817302 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19a5127-b74a-42e1-b70d-b9773250051e service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:29:21.817586 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19a5127-b74a-42e1-b70d-b9773250051e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:29:21.819258 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-e19a5127-b74a-42e1-b70d-b9773250051e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:29:21.926477 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19a5127-b74a-42e1-b70d-b9773250051e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:29:21.927894 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19a5127-b74a-42e1-b70d-b9773250051e service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:29:21.946153 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 732/1461] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:29:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 19:29:24.002811 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c12f1b0-dac3-4763-9fc9-1ea64f012d85 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:29:24.003354 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c12f1b0-dac3-4763-9fc9-1ea64f012d85 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:29:24.003354 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c12f1b0-dac3-4763-9fc9-1ea64f012d85 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:29:24.004788 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 730/1462] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 19:29:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:29:24.010990 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16d7cff-76a9-43bc-ab98-9fe7dab55cbb None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:29:24.011495 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16d7cff-76a9-43bc-ab98-9fe7dab55cbb None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:29:24.011610 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16d7cff-76a9-43bc-ab98-9fe7dab55cbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:29:24.077717 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-e16d7cff-76a9-43bc-ab98-9fe7dab55cbb None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:29:24.078872 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-e16d7cff-76a9-43bc-ab98-9fe7dab55cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:29:24.117526 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:29:24.177188 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 733/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 19:29:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:29:24.217723 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c12f1b0-dac3-4763-9fc9-1ea64f012d85 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:29:24.220370 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-2c12f1b0-dac3-4763-9fc9-1ea64f012d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:29:24.327556 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c12f1b0-dac3-4763-9fc9-1ea64f012d85 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:29:24.370861 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:29:24.407331 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a9824e-3cc8-4e93-9285-6f2eaa2faae6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:29:24.407982 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a9824e-3cc8-4e93-9285-6f2eaa2faae6 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:29:24.408262 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a9824e-3cc8-4e93-9285-6f2eaa2faae6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:29:24.408537 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a9824e-3cc8-4e93-9285-6f2eaa2faae6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:29:24.410255 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-96a9824e-3cc8-4e93-9285-6f2eaa2faae6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:29:24.531800 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a9824e-3cc8-4e93-9285-6f2eaa2faae6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:29:24.533242 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a9824e-3cc8-4e93-9285-6f2eaa2faae6 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:29:24.552892 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 731/1464] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:29:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:29:26.582028 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17817a3-abfc-4b96-ae51-66d6a753c230 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:29:26.582599 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17817a3-abfc-4b96-ae51-66d6a753c230 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:29:26.582599 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17817a3-abfc-4b96-ae51-66d6a753c230 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:29:26.583948 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 734/1465] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 19:29:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:29:26.590012 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0a30d9-ea1c-4ae6-bab5-2d828ec7d557 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:29:26.590480 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0a30d9-ea1c-4ae6-bab5-2d828ec7d557 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:29:26.590580 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0a30d9-ea1c-4ae6-bab5-2d828ec7d557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:29:26.655732 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-cc0a30d9-ea1c-4ae6-bab5-2d828ec7d557 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:29:26.656943 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-cc0a30d9-ea1c-4ae6-bab5-2d828ec7d557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:29:26.674727 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:29:26.753149 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 732/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 19:29:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:29:26.790064 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17817a3-abfc-4b96-ae51-66d6a753c230 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:29:26.791668 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-f17817a3-abfc-4b96-ae51-66d6a753c230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:29:26.894605 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17817a3-abfc-4b96-ae51-66d6a753c230 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:29:26.960140 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:29:26.968032 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae779318-056a-4787-9c33-6c10aa057e2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:29:26.973973 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae779318-056a-4787-9c33-6c10aa057e2b service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:29:26.973973 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae779318-056a-4787-9c33-6c10aa057e2b service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:29:26.973973 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae779318-056a-4787-9c33-6c10aa057e2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:29:26.973973 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-ae779318-056a-4787-9c33-6c10aa057e2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:29:27.079279 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae779318-056a-4787-9c33-6c10aa057e2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:29:27.080706 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae779318-056a-4787-9c33-6c10aa057e2b service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:29:27.098412 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 735/1467] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:29:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:29:29.139682 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efef2c18-a466-4fec-bcb0-804f411bb034 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:29:29.140226 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efef2c18-a466-4fec-bcb0-804f411bb034 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:29:29.140226 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efef2c18-a466-4fec-bcb0-804f411bb034 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:29:29.141627 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 733/1468] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 19:29:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:29:29.147088 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d4da9-6f0c-4d2f-9133-a8c2608f45b2 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:29:29.147511 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d4da9-6f0c-4d2f-9133-a8c2608f45b2 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:29:29.147653 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d4da9-6f0c-4d2f-9133-a8c2608f45b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:29:29.212175 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-772d4da9-6f0c-4d2f-9133-a8c2608f45b2 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:29:29.214020 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-772d4da9-6f0c-4d2f-9133-a8c2608f45b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:29:29.261777 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:29:29.311509 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 736/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 19:29:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:29:29.348047 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efef2c18-a466-4fec-bcb0-804f411bb034 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:29:29.349578 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-efef2c18-a466-4fec-bcb0-804f411bb034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:29:29.463185 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efef2c18-a466-4fec-bcb0-804f411bb034 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:29:29.513149 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:29:29.539578 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35652c4e-3529-4c8c-ae12-3413398c38ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:29:29.540230 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35652c4e-3529-4c8c-ae12-3413398c38ca service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:29:29.540513 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35652c4e-3529-4c8c-ae12-3413398c38ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:29:29.540839 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35652c4e-3529-4c8c-ae12-3413398c38ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:29:29.542576 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-35652c4e-3529-4c8c-ae12-3413398c38ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:29:29.651736 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35652c4e-3529-4c8c-ae12-3413398c38ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:29:29.653175 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35652c4e-3529-4c8c-ae12-3413398c38ca service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:29:29.671380 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 734/1470] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:29:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:29:39.309680 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=22 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:29:39.669065 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=66 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:29:49.259614 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb8fd3-e9f2-494d-bded-9b3b8e84ceb5 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:29:49.260194 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb8fd3-e9f2-494d-bded-9b3b8e84ceb5 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:29:49.260194 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb8fd3-e9f2-494d-bded-9b3b8e84ceb5 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:29:49.261584 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 737/1471] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:29:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:29:49.267016 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ef74d-a13d-4d92-b985-5d5e84e340dc None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:29:49.267426 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ef74d-a13d-4d92-b985-5d5e84e340dc None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:29:49.267581 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ef74d-a13d-4d92-b985-5d5e84e340dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:29:49.334405 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-d90ef74d-a13d-4d92-b985-5d5e84e340dc None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:29:49.335659 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d90ef74d-a13d-4d92-b985-5d5e84e340dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:29:49.434923 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 735/1472] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:29:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:29:49.483086 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e5c637-9c27-43d2-89c7-d6a63d850637 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:29:49.483383 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e5c637-9c27-43d2-89c7-d6a63d850637 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:29:49.483657 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e5c637-9c27-43d2-89c7-d6a63d850637 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:29:49.485053 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 738/1473] 10.222.0.33 () {54 vars in 960 bytes} [Fri Jul 15 19:29:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:29:49.489940 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2c555e-1366-4982-9f37-19c39a0b9502 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:29:49.490231 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2c555e-1366-4982-9f37-19c39a0b9502 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:29:49.490507 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2c555e-1366-4982-9f37-19c39a0b9502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:29:49.565196 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-bf2c555e-1366-4982-9f37-19c39a0b9502 None None] MFA Rules not processed for user `04b5b06b0015450a95a28bbcc757cc8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:29:49.566452 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-bf2c555e-1366-4982-9f37-19c39a0b9502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:29:49.587490 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:29:49.642767 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 736/1474] 10.222.0.33 () {58 vars in 1057 bytes} [Fri Jul 15 19:29:49 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:29:49.651029 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e5c637-9c27-43d2-89c7-d6a63d850637 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:29:49.652540 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-51e5c637-9c27-43d2-89c7-d6a63d850637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:29:49.755169 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e5c637-9c27-43d2-89c7-d6a63d850637 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:29:49.829605 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b85d8c-4ed8-4cc2-b51b-85b97774beaf service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04b5b06b0015450a95a28bbcc757cc8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:29:49.830296 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b85d8c-4ed8-4cc2-b51b-85b97774beaf service glance] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:29:49.830581 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b85d8c-4ed8-4cc2-b51b-85b97774beaf service glance] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:29:49.830858 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b85d8c-4ed8-4cc2-b51b-85b97774beaf service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:29:49.832545 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-29b85d8c-4ed8-4cc2-b51b-85b97774beaf service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:29:49.886956 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:29:49.943112 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b85d8c-4ed8-4cc2-b51b-85b97774beaf service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:29:49.944581 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b85d8c-4ed8-4cc2-b51b-85b97774beaf service glance] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:29:49.962907 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 739/1475] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:29:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:29:52.115716 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22ef0d0-6b39-48ca-9b00-803bde0f815e None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:29:52.116268 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22ef0d0-6b39-48ca-9b00-803bde0f815e None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:29:52.116268 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22ef0d0-6b39-48ca-9b00-803bde0f815e None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:29:52.117606 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 737/1476] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:29:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:29:52.122737 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87836c2e-3ee6-46a2-9e90-ee6ff73bda97 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:29:52.123135 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87836c2e-3ee6-46a2-9e90-ee6ff73bda97 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:29:52.123283 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87836c2e-3ee6-46a2-9e90-ee6ff73bda97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:29:52.187746 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-87836c2e-3ee6-46a2-9e90-ee6ff73bda97 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:29:52.188957 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-87836c2e-3ee6-46a2-9e90-ee6ff73bda97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:29:52.283524 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 740/1477] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:29:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:29:52.307858 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22ef0d0-6b39-48ca-9b00-803bde0f815e None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:29:52.309412 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d22ef0d0-6b39-48ca-9b00-803bde0f815e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:29:52.433309 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22ef0d0-6b39-48ca-9b00-803bde0f815e None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:29:52.482539 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:29:52.536159 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b07f969-ff40-425a-912f-97f27cc0a8d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:29:52.536815 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b07f969-ff40-425a-912f-97f27cc0a8d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:29:52.537096 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b07f969-ff40-425a-912f-97f27cc0a8d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:29:52.537368 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b07f969-ff40-425a-912f-97f27cc0a8d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:29:52.538127 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b07f969-ff40-425a-912f-97f27cc0a8d5 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:29:52.539956 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: WARNING py.warnings [None req-4b07f969-ff40-425a-912f-97f27cc0a8d5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 15 19:29:52.539956 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: warnings.warn(msg) Jul 15 19:29:52.539956 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: Jul 15 19:29:52.541620 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b07f969-ff40-425a-912f-97f27cc0a8d5 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:29:52.560083 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 738/1478] 10.222.0.33 () {56 vars in 1108 bytes} [Fri Jul 15 19:29:52 2022] GET /identity/v3/projects => generated 2553 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:29:54.582538 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f184893e-6593-4646-a343-4468264debe5 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:29:54.583107 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f184893e-6593-4646-a343-4468264debe5 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:29:54.583107 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f184893e-6593-4646-a343-4468264debe5 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:29:54.584466 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 741/1479] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:29:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:29:54.589739 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2639a5a6-cc0c-4d2c-a7c2-31634b08eb6b None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:29:54.590212 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2639a5a6-cc0c-4d2c-a7c2-31634b08eb6b None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:29:54.590397 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2639a5a6-cc0c-4d2c-a7c2-31634b08eb6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:29:54.655865 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-2639a5a6-cc0c-4d2c-a7c2-31634b08eb6b None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:29:54.657072 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-2639a5a6-cc0c-4d2c-a7c2-31634b08eb6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:29:54.752859 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 739/1480] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:29:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:29:54.796448 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f184893e-6593-4646-a343-4468264debe5 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:29:54.798016 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-f184893e-6593-4646-a343-4468264debe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:29:54.808799 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:29:54.946874 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f184893e-6593-4646-a343-4468264debe5 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:29:55.024891 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:29:55.057442 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decfeb26-72bf-446f-aea9-6dee81558023 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7673a3b5c0324e07a9535ac70ed7a491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:29:55.058121 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decfeb26-72bf-446f-aea9-6dee81558023 service nova] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:29:55.058402 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decfeb26-72bf-446f-aea9-6dee81558023 service nova] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:29:55.058673 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decfeb26-72bf-446f-aea9-6dee81558023 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:29:55.060311 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-decfeb26-72bf-446f-aea9-6dee81558023 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:29:55.148113 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decfeb26-72bf-446f-aea9-6dee81558023 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:29:55.149745 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decfeb26-72bf-446f-aea9-6dee81558023 service nova] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:29:55.168775 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 742/1481] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:29:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:29:57.342049 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716a8273-cf39-4fde-b677-897e53edddba None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:29:57.342647 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716a8273-cf39-4fde-b677-897e53edddba None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:29:57.342647 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716a8273-cf39-4fde-b677-897e53edddba None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:29:57.343990 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 740/1482] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:29:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:29:57.349076 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c525002-674c-4b02-aea5-e233acddff5c None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:29:57.349483 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c525002-674c-4b02-aea5-e233acddff5c None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:29:57.349627 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c525002-674c-4b02-aea5-e233acddff5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:29:57.412754 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-5c525002-674c-4b02-aea5-e233acddff5c None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:29:57.413987 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-5c525002-674c-4b02-aea5-e233acddff5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:29:57.458568 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:29:57.507338 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 743/1483] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:29:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:29:57.543962 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716a8273-cf39-4fde-b677-897e53edddba None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:29:57.545568 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-716a8273-cf39-4fde-b677-897e53edddba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:29:57.569042 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:29:57.649227 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716a8273-cf39-4fde-b677-897e53edddba None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:29:57.720834 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687593d3-f1a5-4d34-aad9-316ef8140fb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:29:57.721404 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687593d3-f1a5-4d34-aad9-316ef8140fb1 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:29:57.721683 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687593d3-f1a5-4d34-aad9-316ef8140fb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:29:57.721980 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687593d3-f1a5-4d34-aad9-316ef8140fb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:29:57.723616 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-687593d3-f1a5-4d34-aad9-316ef8140fb1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:29:57.798877 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:29:57.834623 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687593d3-f1a5-4d34-aad9-316ef8140fb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:29:57.836073 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687593d3-f1a5-4d34-aad9-316ef8140fb1 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:29:57.853991 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 741/1484] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:29:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:29:59.898014 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256dfb6e-dd02-4c53-b9c2-07a164affa03 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:29:59.898581 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256dfb6e-dd02-4c53-b9c2-07a164affa03 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:29:59.898581 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256dfb6e-dd02-4c53-b9c2-07a164affa03 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:29:59.899855 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 744/1485] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:29:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:29:59.905031 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b8c70-b93f-468c-a91f-5abefbeb2b37 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:29:59.905491 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b8c70-b93f-468c-a91f-5abefbeb2b37 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:29:59.905625 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b8c70-b93f-468c-a91f-5abefbeb2b37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:29:59.970964 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-2c7b8c70-b93f-468c-a91f-5abefbeb2b37 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:29:59.972220 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-2c7b8c70-b93f-468c-a91f-5abefbeb2b37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:30:00.069139 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 742/1486] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:29:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:30:00.106665 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256dfb6e-dd02-4c53-b9c2-07a164affa03 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:30:00.108184 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-256dfb6e-dd02-4c53-b9c2-07a164affa03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:30:00.213755 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256dfb6e-dd02-4c53-b9c2-07a164affa03 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:30:00.284122 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:30:00.285969 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d345c9-c877-4bf0-9c77-f224b0410152 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:30:00.286621 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d345c9-c877-4bf0-9c77-f224b0410152 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:30:00.286904 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d345c9-c877-4bf0-9c77-f224b0410152 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:30:00.287177 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d345c9-c877-4bf0-9c77-f224b0410152 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:30:00.288835 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-97d345c9-c877-4bf0-9c77-f224b0410152 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:30:00.412422 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d345c9-c877-4bf0-9c77-f224b0410152 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:30:00.413903 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d345c9-c877-4bf0-9c77-f224b0410152 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:30:00.431498 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 745/1487] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:30:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:30:02.533284 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102e8cce-ca45-4fa0-8511-46e009a83ca6 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:30:02.533889 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102e8cce-ca45-4fa0-8511-46e009a83ca6 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:30:02.533889 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102e8cce-ca45-4fa0-8511-46e009a83ca6 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:30:02.535233 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 743/1488] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:30:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:30:02.540980 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ae409-7e2c-44d2-a380-e3b613568997 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:30:02.541390 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ae409-7e2c-44d2-a380-e3b613568997 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:30:02.541526 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ae409-7e2c-44d2-a380-e3b613568997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:30:02.604924 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-cb4ae409-7e2c-44d2-a380-e3b613568997 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:30:02.606406 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-cb4ae409-7e2c-44d2-a380-e3b613568997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:30:02.659213 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:30:02.716049 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 746/1489] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:30:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:30:02.757368 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102e8cce-ca45-4fa0-8511-46e009a83ca6 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:30:02.759050 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-102e8cce-ca45-4fa0-8511-46e009a83ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:30:02.795660 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:30:02.863232 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102e8cce-ca45-4fa0-8511-46e009a83ca6 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:30:02.934767 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fdef83-fce5-481e-a500-8fe1c2302918 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:30:02.935409 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fdef83-fce5-481e-a500-8fe1c2302918 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:30:02.935687 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fdef83-fce5-481e-a500-8fe1c2302918 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:30:02.935960 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fdef83-fce5-481e-a500-8fe1c2302918 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:30:02.937595 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-00fdef83-fce5-481e-a500-8fe1c2302918 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:30:03.017082 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:30:03.052610 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fdef83-fce5-481e-a500-8fe1c2302918 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:30:03.053997 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fdef83-fce5-481e-a500-8fe1c2302918 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:30:03.072412 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 744/1490] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:30:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:30:05.527577 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e4e399-456d-4d0b-a713-a86eee940f94 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:30:05.528138 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e4e399-456d-4d0b-a713-a86eee940f94 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:30:05.528138 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e4e399-456d-4d0b-a713-a86eee940f94 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:30:05.529406 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 747/1491] 10.222.0.33 () {54 vars in 926 bytes} [Fri Jul 15 19:30:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:30:05.534750 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7e526f-e4a8-4f71-a281-4cb2d31e8f25 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:30:05.535180 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7e526f-e4a8-4f71-a281-4cb2d31e8f25 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:30:05.535292 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7e526f-e4a8-4f71-a281-4cb2d31e8f25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:30:05.599075 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-3a7e526f-e4a8-4f71-a281-4cb2d31e8f25 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:30:05.600295 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-3a7e526f-e4a8-4f71-a281-4cb2d31e8f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:30:05.697240 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 745/1492] 10.222.0.33 () {58 vars in 1023 bytes} [Fri Jul 15 19:30:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:30:05.737516 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e4e399-456d-4d0b-a713-a86eee940f94 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:30:05.739103 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-d8e4e399-456d-4d0b-a713-a86eee940f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:30:05.848660 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e4e399-456d-4d0b-a713-a86eee940f94 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:30:05.921015 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b8399d-bdfd-4f2f-a1ff-55af181a92f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:30:05.921653 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b8399d-bdfd-4f2f-a1ff-55af181a92f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:30:05.921975 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b8399d-bdfd-4f2f-a1ff-55af181a92f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:30:05.922251 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b8399d-bdfd-4f2f-a1ff-55af181a92f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:30:05.923876 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-f9b8399d-bdfd-4f2f-a1ff-55af181a92f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:30:05.933228 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:30:06.031843 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b8399d-bdfd-4f2f-a1ff-55af181a92f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:30:06.033269 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b8399d-bdfd-4f2f-a1ff-55af181a92f3 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:30:06.053067 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 748/1493] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:30:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:30:15.694879 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=82 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:30:16.050489 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=50 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:31:43.591870 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3803626-45de-4c85-a451-eed9a2b29bed None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:31:43.592458 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3803626-45de-4c85-a451-eed9a2b29bed None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:31:43.592458 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3803626-45de-4c85-a451-eed9a2b29bed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:31:43.672412 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-d3803626-45de-4c85-a451-eed9a2b29bed None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:31:43.673607 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d3803626-45de-4c85-a451-eed9a2b29bed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:31:43.770211 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 746/1494] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:31:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:31:43.778469 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57823995-74b7-423d-af10-b07fdda03f4f None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:31:43.778986 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57823995-74b7-423d-af10-b07fdda03f4f None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:31:43.779108 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57823995-74b7-423d-af10-b07fdda03f4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:31:43.853525 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-57823995-74b7-423d-af10-b07fdda03f4f None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:31:43.854726 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-57823995-74b7-423d-af10-b07fdda03f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:31:43.949732 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 749/1495] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:31:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:31:43.954522 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3803626-45de-4c85-a451-eed9a2b29bed None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:31:43.955730 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d3803626-45de-4c85-a451-eed9a2b29bed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:31:43.966990 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3803626-45de-4c85-a451-eed9a2b29bed None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:31:44.028190 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:31:44.061532 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e119b675-541b-484f-8373-fd350f091b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:31:44.072159 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e119b675-541b-484f-8373-fd350f091b90 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:31:44.072159 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e119b675-541b-484f-8373-fd350f091b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:31:44.072159 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e119b675-541b-484f-8373-fd350f091b90 admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:31:44.072159 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e119b675-541b-484f-8373-fd350f091b90 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:31:44.072159 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e119b675-541b-484f-8373-fd350f091b90 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:31:44.072616 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: WARNING py.warnings [None req-e119b675-541b-484f-8373-fd350f091b90 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 15 19:31:44.072616 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: warnings.warn(msg) Jul 15 19:31:44.072616 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: Jul 15 19:31:44.072616 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e119b675-541b-484f-8373-fd350f091b90 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:31:44.077032 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 747/1496] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jul 15 19:31:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:31:44.085399 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57823995-74b7-423d-af10-b07fdda03f4f None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:31:44.094867 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57823995-74b7-423d-af10-b07fdda03f4f None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:31:44.154789 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:31:44.188908 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5720d8-665f-4689-a9d6-8d3b9e1879a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:31:44.189570 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5720d8-665f-4689-a9d6-8d3b9e1879a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:31:44.189889 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5720d8-665f-4689-a9d6-8d3b9e1879a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:31:44.190170 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5720d8-665f-4689-a9d6-8d3b9e1879a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:31:44.190872 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5720d8-665f-4689-a9d6-8d3b9e1879a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:31:44.192647 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: WARNING py.warnings [None req-8e5720d8-665f-4689-a9d6-8d3b9e1879a8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 15 19:31:44.192647 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: warnings.warn(msg) Jul 15 19:31:44.192647 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: Jul 15 19:31:44.193868 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5720d8-665f-4689-a9d6-8d3b9e1879a8 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:31:44.319551 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 750/1497] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jul 15 19:31:44 2022] POST /identity/v3/projects => generated 379 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:31:44.323770 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e119b675-541b-484f-8373-fd350f091b90 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:31:44.325307 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e119b675-541b-484f-8373-fd350f091b90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:31:44.358471 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e119b675-541b-484f-8373-fd350f091b90 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:31:44.458429 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8fc867-9432-4adc-b2c6-a25cb76740af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:31:44.459021 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8fc867-9432-4adc-b2c6-a25cb76740af admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:31:44.459381 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8fc867-9432-4adc-b2c6-a25cb76740af admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:31:44.459816 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8fc867-9432-4adc-b2c6-a25cb76740af admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:31:44.460521 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8fc867-9432-4adc-b2c6-a25cb76740af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:31:44.461535 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: WARNING py.warnings [None req-2a8fc867-9432-4adc-b2c6-a25cb76740af admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 15 19:31:44.461535 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: warnings.warn(msg) Jul 15 19:31:44.461535 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: Jul 15 19:31:44.462778 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8fc867-9432-4adc-b2c6-a25cb76740af admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:31:44.569634 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 748/1498] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jul 15 19:31:44 2022] POST /identity/v3/users => generated 290 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:31:44.573665 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5720d8-665f-4689-a9d6-8d3b9e1879a8 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:31:44.575219 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-8e5720d8-665f-4689-a9d6-8d3b9e1879a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:31:44.637653 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5720d8-665f-4689-a9d6-8d3b9e1879a8 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:31:44.722361 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:31:44.733595 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b58dec-b05f-4b01-990a-11e286c1a1d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:31:44.734329 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b58dec-b05f-4b01-990a-11e286c1a1d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:31:44.734615 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b58dec-b05f-4b01-990a-11e286c1a1d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:31:44.734890 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b58dec-b05f-4b01-990a-11e286c1a1d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:31:44.735636 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b58dec-b05f-4b01-990a-11e286c1a1d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:31:44.736662 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: WARNING py.warnings [None req-e2b58dec-b05f-4b01-990a-11e286c1a1d5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 15 19:31:44.736662 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: warnings.warn(msg) Jul 15 19:31:44.736662 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: Jul 15 19:31:44.737570 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b58dec-b05f-4b01-990a-11e286c1a1d5 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:31:44.751366 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 751/1499] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:31:44 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:31:44.755493 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8fc867-9432-4adc-b2c6-a25cb76740af admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:31:44.757077 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-2a8fc867-9432-4adc-b2c6-a25cb76740af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:31:44.768779 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8fc867-9432-4adc-b2c6-a25cb76740af admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:31:44.795036 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:31:44.867099 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabc174a-2dd0-45f8-93f6-8f6e7ee1b08c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:31:44.867775 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabc174a-2dd0-45f8-93f6-8f6e7ee1b08c admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:31:44.868056 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabc174a-2dd0-45f8-93f6-8f6e7ee1b08c admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:31:44.868331 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabc174a-2dd0-45f8-93f6-8f6e7ee1b08c admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:31:44.869125 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabc174a-2dd0-45f8-93f6-8f6e7ee1b08c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:31:44.870196 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: WARNING py.warnings [None req-dabc174a-2dd0-45f8-93f6-8f6e7ee1b08c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 15 19:31:44.870196 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: warnings.warn(msg) Jul 15 19:31:44.870196 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: Jul 15 19:31:44.871081 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabc174a-2dd0-45f8-93f6-8f6e7ee1b08c admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:31:44.883602 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 749/1500] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:31:44 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:31:44.887809 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b58dec-b05f-4b01-990a-11e286c1a1d5 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:31:44.889351 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-e2b58dec-b05f-4b01-990a-11e286c1a1d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:31:44.922878 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b58dec-b05f-4b01-990a-11e286c1a1d5 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:31:45.014960 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7ca651-29de-469a-9cc2-334c1dacc3eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:31:45.015769 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7ca651-29de-469a-9cc2-334c1dacc3eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:31:45.016046 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7ca651-29de-469a-9cc2-334c1dacc3eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:31:45.016316 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7ca651-29de-469a-9cc2-334c1dacc3eb admin admin] PATH_INFO: `/v3/projects/84e4cb74aace408ca0a51207449a8a99/users/3b066c00e0d7401289d51f4d0aa1f661/roles/28763c6e70094116b3a4b2895895fa58` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:31:45.038383 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7ca651-29de-469a-9cc2-334c1dacc3eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=84e4cb74aace408ca0a51207449a8a99, user_id=3b066c00e0d7401289d51f4d0aa1f661, role_id=28763c6e70094116b3a4b2895895fa58)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:31:45.039388 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: WARNING py.warnings [None req-8a7ca651-29de-469a-9cc2-334c1dacc3eb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 15 19:31:45.039388 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: warnings.warn(msg) Jul 15 19:31:45.039388 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: Jul 15 19:31:45.041306 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7ca651-29de-469a-9cc2-334c1dacc3eb admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:31:45.147119 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 752/1501] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:31:44 2022] PUT /identity/v3/projects/84e4cb74aace408ca0a51207449a8a99/users/3b066c00e0d7401289d51f4d0aa1f661/roles/28763c6e70094116b3a4b2895895fa58 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:31:45.159202 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d776261-d669-4f47-9d9b-ade1920d1e28 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:31:45.159499 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d776261-d669-4f47-9d9b-ade1920d1e28 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:31:45.159780 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d776261-d669-4f47-9d9b-ade1920d1e28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:31:45.333531 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-9d776261-d669-4f47-9d9b-ade1920d1e28 None None] MFA Rules not processed for user `3b066c00e0d7401289d51f4d0aa1f661`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:31:45.334788 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9d776261-d669-4f47-9d9b-ade1920d1e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:31:45.412343 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 750/1502] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:31:45 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:31:45.425553 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98225d2c-eaea-4ec5-bcc8-06c65a07d7c5 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:31:45.425874 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98225d2c-eaea-4ec5-bcc8-06c65a07d7c5 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:31:45.426154 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98225d2c-eaea-4ec5-bcc8-06c65a07d7c5 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:31:45.427543 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 753/1503] 10.222.0.33 () {54 vars in 963 bytes} [Fri Jul 15 19:31:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:31:45.432542 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba59834-d049-440c-8ad1-59864f71bfab None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:31:45.432830 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba59834-d049-440c-8ad1-59864f71bfab None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:31:45.436652 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba59834-d049-440c-8ad1-59864f71bfab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:31:45.462495 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:31:45.505993 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-5ba59834-d049-440c-8ad1-59864f71bfab None None] MFA Rules not processed for user `c86c2df3d378490ca7ea497a73c19d40`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:31:45.507360 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-5ba59834-d049-440c-8ad1-59864f71bfab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:31:45.587750 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 751/1504] 10.222.0.33 () {58 vars in 1060 bytes} [Fri Jul 15 19:31:45 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:31:45.595355 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98225d2c-eaea-4ec5-bcc8-06c65a07d7c5 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:31:45.596857 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-98225d2c-eaea-4ec5-bcc8-06c65a07d7c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:31:45.651132 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:31:45.713709 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98225d2c-eaea-4ec5-bcc8-06c65a07d7c5 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:31:45.793350 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ac2b20-b452-496c-8430-323bfcdce458 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86c2df3d378490ca7ea497a73c19d40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:31:45.794036 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ac2b20-b452-496c-8430-323bfcdce458 service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:31:45.794317 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ac2b20-b452-496c-8430-323bfcdce458 service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:31:45.794592 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ac2b20-b452-496c-8430-323bfcdce458 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:31:45.796239 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-d6ac2b20-b452-496c-8430-323bfcdce458 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:31:45.896952 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ac2b20-b452-496c-8430-323bfcdce458 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:31:45.898347 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ac2b20-b452-496c-8430-323bfcdce458 service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:31:45.901879 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:31:45.915856 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 754/1505] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:31:45 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:31:45.958927 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78336d68-a0b3-46e7-98d1-15b3fbe01cf9 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:31:45.959358 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78336d68-a0b3-46e7-98d1-15b3fbe01cf9 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:31:45.959466 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78336d68-a0b3-46e7-98d1-15b3fbe01cf9 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:31:45.960855 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 752/1506] 10.222.0.33 () {52 vars in 829 bytes} [Fri Jul 15 19:31:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:31:45.964308 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ac2b20-b452-496c-8430-323bfcdce458 service cinder] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:31:45.965558 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-d6ac2b20-b452-496c-8430-323bfcdce458 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:31:46.070788 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ac2b20-b452-496c-8430-323bfcdce458 service cinder] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:31:46.163611 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157c51e5-d8d5-4ce7-b262-d98c69973446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:31:46.164403 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157c51e5-d8d5-4ce7-b262-d98c69973446 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:31:46.164686 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157c51e5-d8d5-4ce7-b262-d98c69973446 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:31:46.164963 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157c51e5-d8d5-4ce7-b262-d98c69973446 admin admin] PATH_INFO: `/v3/users/3b066c00e0d7401289d51f4d0aa1f661` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:31:46.177619 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157c51e5-d8d5-4ce7-b262-d98c69973446 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b066c00e0d7401289d51f4d0aa1f661)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:31:46.178665 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: WARNING py.warnings [None req-157c51e5-d8d5-4ce7-b262-d98c69973446 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 15 19:31:46.178665 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: warnings.warn(msg) Jul 15 19:31:46.178665 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: Jul 15 19:31:46.179875 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157c51e5-d8d5-4ce7-b262-d98c69973446 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:31:46.189397 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:31:46.444152 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-157c51e5-d8d5-4ce7-b262-d98c69973446 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b066c00e0d7401289d51f4d0aa1f661'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:31:46.462120 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-157c51e5-d8d5-4ce7-b262-d98c69973446 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b066c00e0d7401289d51f4d0aa1f661'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:31:46.545876 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-157c51e5-d8d5-4ce7-b262-d98c69973446 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b066c00e0d7401289d51f4d0aa1f661'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:31:46.546337 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-157c51e5-d8d5-4ce7-b262-d98c69973446 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b066c00e0d7401289d51f4d0aa1f661'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:31:46.546866 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-157c51e5-d8d5-4ce7-b262-d98c69973446 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b066c00e0d7401289d51f4d0aa1f661'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:31:46.566398 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 755/1507] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jul 15 19:31:45 2022] DELETE /identity/v3/users/3b066c00e0d7401289d51f4d0aa1f661 => generated 0 bytes in 603 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:31:46.570865 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a243dc09-d804-44a1-a7ff-a0058a6e1ef3 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:31:46.571144 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a243dc09-d804-44a1-a7ff-a0058a6e1ef3 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:31:46.571416 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a243dc09-d804-44a1-a7ff-a0058a6e1ef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:31:46.625363 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-a243dc09-d804-44a1-a7ff-a0058a6e1ef3 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:31:46.626605 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-a243dc09-d804-44a1-a7ff-a0058a6e1ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:31:46.706247 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:31:46.718656 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 753/1508] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:31:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:31:46.728848 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157c51e5-d8d5-4ce7-b262-d98c69973446 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:31:46.730142 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-157c51e5-d8d5-4ce7-b262-d98c69973446 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:31:46.830115 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157c51e5-d8d5-4ce7-b262-d98c69973446 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:31:46.885892 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:31:46.904963 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f4bd05-5ac6-4e3f-82c0-e246db9c5193 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:31:46.906074 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f4bd05-5ac6-4e3f-82c0-e246db9c5193 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:31:46.906074 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f4bd05-5ac6-4e3f-82c0-e246db9c5193 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:31:46.906277 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f4bd05-5ac6-4e3f-82c0-e246db9c5193 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:31:46.907739 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-70f4bd05-5ac6-4e3f-82c0-e246db9c5193 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:31:47.017135 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f4bd05-5ac6-4e3f-82c0-e246db9c5193 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:31:47.018779 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f4bd05-5ac6-4e3f-82c0-e246db9c5193 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:31:47.038011 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 756/1509] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:31:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:31:47.502175 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a243dc09-d804-44a1-a7ff-a0058a6e1ef3 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:31:47.503418 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-a243dc09-d804-44a1-a7ff-a0058a6e1ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:31:47.514790 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a243dc09-d804-44a1-a7ff-a0058a6e1ef3 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:31:47.616002 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fa5712-82c4-442c-bd1a-b017721f2164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:31:47.616858 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fa5712-82c4-442c-bd1a-b017721f2164 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:31:47.617133 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fa5712-82c4-442c-bd1a-b017721f2164 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:31:47.617425 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fa5712-82c4-442c-bd1a-b017721f2164 admin admin] PATH_INFO: `/v3/projects/84e4cb74aace408ca0a51207449a8a99` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:31:47.627801 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fa5712-82c4-442c-bd1a-b017721f2164 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84e4cb74aace408ca0a51207449a8a99)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:31:47.628818 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: WARNING py.warnings [None req-99fa5712-82c4-442c-bd1a-b017721f2164 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 15 19:31:47.628818 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: warnings.warn(msg) Jul 15 19:31:47.628818 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: Jul 15 19:31:47.630123 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fa5712-82c4-442c-bd1a-b017721f2164 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:31:47.786953 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-99fa5712-82c4-442c-bd1a-b017721f2164 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84e4cb74aace408ca0a51207449a8a99', 'request_id': 'req-99fa5712-82c4-442c-bd1a-b017721f2164'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:31:47.889350 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-99fa5712-82c4-442c-bd1a-b017721f2164 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84e4cb74aace408ca0a51207449a8a99', 'request_id': 'req-99fa5712-82c4-442c-bd1a-b017721f2164'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:31:47.897549 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-99fa5712-82c4-442c-bd1a-b017721f2164 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84e4cb74aace408ca0a51207449a8a99', 'request_id': 'req-99fa5712-82c4-442c-bd1a-b017721f2164'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:31:47.927707 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-99fa5712-82c4-442c-bd1a-b017721f2164 admin admin] The token cache is being invalidate because project 84e4cb74aace408ca0a51207449a8a99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:31:47.928304 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-99fa5712-82c4-442c-bd1a-b017721f2164 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:31:47.928865 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-99fa5712-82c4-442c-bd1a-b017721f2164 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:31:47.930292 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 754/1510] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jul 15 19:31:47 2022] DELETE /identity/v3/projects/84e4cb74aace408ca0a51207449a8a99 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:31:57.035137 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=62 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:31:57.925468 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:33:28.003503 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d3990a-b69b-4d55-ae62-8650eb4e5fa2 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:33:28.004094 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d3990a-b69b-4d55-ae62-8650eb4e5fa2 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:33:28.004094 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d3990a-b69b-4d55-ae62-8650eb4e5fa2 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:33:28.005443 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 757/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 15 19:33:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:33:28.010228 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42589f5-a536-4b33-9fa7-f22714e23ff3 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:33:28.010679 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42589f5-a536-4b33-9fa7-f22714e23ff3 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:33:28.010792 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42589f5-a536-4b33-9fa7-f22714e23ff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:33:28.088352 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-a42589f5-a536-4b33-9fa7-f22714e23ff3 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:33:28.089575 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-a42589f5-a536-4b33-9fa7-f22714e23ff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:33:28.194634 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 755/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 15 19:33:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:33:28.349631 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d3990a-b69b-4d55-ae62-8650eb4e5fa2 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:33:28.352008 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-c2d3990a-b69b-4d55-ae62-8650eb4e5fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:33:28.469086 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d3990a-b69b-4d55-ae62-8650eb4e5fa2 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:33:28.474826 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8394432a-85f9-4cbd-ae61-84e08c2572fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7673a3b5c0324e07a9535ac70ed7a491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:33:28.475482 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8394432a-85f9-4cbd-ae61-84e08c2572fe service nova] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:33:28.475766 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8394432a-85f9-4cbd-ae61-84e08c2572fe service nova] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:33:28.476042 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8394432a-85f9-4cbd-ae61-84e08c2572fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:33:28.484648 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8394432a-85f9-4cbd-ae61-84e08c2572fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:33:28.486269 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8394432a-85f9-4cbd-ae61-84e08c2572fe service nova] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:33:28.495887 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 758/1513] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:33:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:33:29.343149 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb079643-50f9-405b-b3a4-c134fec3da74 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:33:29.343714 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb079643-50f9-405b-b3a4-c134fec3da74 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:33:29.343714 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb079643-50f9-405b-b3a4-c134fec3da74 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:33:29.345174 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 756/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 15 19:33:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:33:29.350772 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc5a289-f91e-4b7f-9b09-00df96959272 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:33:29.351252 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc5a289-f91e-4b7f-9b09-00df96959272 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:33:29.351363 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc5a289-f91e-4b7f-9b09-00df96959272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:33:29.373165 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-bcc5a289-f91e-4b7f-9b09-00df96959272 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:33:29.374434 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-bcc5a289-f91e-4b7f-9b09-00df96959272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:33:29.383486 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 759/1515] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 15 19:33:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:33:29.537950 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb079643-50f9-405b-b3a4-c134fec3da74 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:33:29.552173 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb079643-50f9-405b-b3a4-c134fec3da74 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:33:29.557639 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47ffcd1-f279-4693-a7d8-adb8d71e57c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7673a3b5c0324e07a9535ac70ed7a491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:33:29.558315 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47ffcd1-f279-4693-a7d8-adb8d71e57c7 service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:33:29.558599 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47ffcd1-f279-4693-a7d8-adb8d71e57c7 service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:33:29.558875 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47ffcd1-f279-4693-a7d8-adb8d71e57c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:33:29.567175 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47ffcd1-f279-4693-a7d8-adb8d71e57c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:33:29.568757 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47ffcd1-f279-4693-a7d8-adb8d71e57c7 service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:33:29.578613 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 757/1516] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:33:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:33:30.231695 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc848cc3-baa0-4e8a-a918-7812e0315d97 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:33:30.231890 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc848cc3-baa0-4e8a-a918-7812e0315d97 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:33:30.232278 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc848cc3-baa0-4e8a-a918-7812e0315d97 None None] PATH_INFO: `/v3` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:33:30.233603 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 760/1517] 10.222.0.30 () {54 vars in 925 bytes} [Fri Jul 15 19:33:30 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:33:30.260794 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8acb1c-b26c-4473-b6ae-0714f1b6b04c None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:33:30.261080 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8acb1c-b26c-4473-b6ae-0714f1b6b04c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:33:30.261362 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8acb1c-b26c-4473-b6ae-0714f1b6b04c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:33:30.318051 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-2f8acb1c-b26c-4473-b6ae-0714f1b6b04c None None] MFA Rules not processed for user `52a0fce231ba4785a685ac9898d07503`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:33:30.319239 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-2f8acb1c-b26c-4473-b6ae-0714f1b6b04c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:33:30.335954 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:33:30.389656 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 758/1518] 10.222.0.30 () {58 vars in 1015 bytes} [Fri Jul 15 19:33:30 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:33:39.367721 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=56 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:33:40.028730 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a377d18-a78a-47a9-b4af-8b9af99da15c None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:33:40.029003 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a377d18-a78a-47a9-b4af-8b9af99da15c None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:33:40.029319 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a377d18-a78a-47a9-b4af-8b9af99da15c None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:33:40.030746 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 761/1519] 10.222.0.33 () {54 vars in 960 bytes} [Fri Jul 15 19:33:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:33:40.035943 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d197fa41-6f54-4200-89a9-26845544331c None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:33:40.036433 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d197fa41-6f54-4200-89a9-26845544331c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:33:40.036541 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d197fa41-6f54-4200-89a9-26845544331c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:33:40.055075 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-d197fa41-6f54-4200-89a9-26845544331c None None] MFA Rules not processed for user `52a0fce231ba4785a685ac9898d07503`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:33:40.056283 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d197fa41-6f54-4200-89a9-26845544331c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:33:40.064841 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 759/1520] 10.222.0.33 () {58 vars in 1057 bytes} [Fri Jul 15 19:33:40 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:33:40.072986 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a377d18-a78a-47a9-b4af-8b9af99da15c None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:33:40.087455 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a377d18-a78a-47a9-b4af-8b9af99da15c None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:33:40.093463 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948279ae-9ac8-4abe-88a3-c20e9b573adb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52a0fce231ba4785a685ac9898d07503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:33:40.094178 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948279ae-9ac8-4abe-88a3-c20e9b573adb service placement] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:33:40.094467 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948279ae-9ac8-4abe-88a3-c20e9b573adb service placement] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:33:40.094747 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948279ae-9ac8-4abe-88a3-c20e9b573adb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:33:40.103812 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948279ae-9ac8-4abe-88a3-c20e9b573adb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:33:40.105388 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948279ae-9ac8-4abe-88a3-c20e9b573adb service placement] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:33:40.115816 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 762/1521] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:33:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 19:33:40.828711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf2566b-462e-449a-afed-c2077610f312 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:33:40.831658 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf2566b-462e-449a-afed-c2077610f312 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:33:40.831658 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf2566b-462e-449a-afed-c2077610f312 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:33:40.831658 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 760/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 15 19:33:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:33:40.837184 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113fabde-9e88-4a9a-8cc7-ad477e708137 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:33:40.837628 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113fabde-9e88-4a9a-8cc7-ad477e708137 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:33:40.837733 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113fabde-9e88-4a9a-8cc7-ad477e708137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:33:40.856237 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-113fabde-9e88-4a9a-8cc7-ad477e708137 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:33:40.857444 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-113fabde-9e88-4a9a-8cc7-ad477e708137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:33:40.865900 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 763/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 15 19:33:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:33:41.030823 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf2566b-462e-449a-afed-c2077610f312 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:33:41.036549 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf2566b-462e-449a-afed-c2077610f312 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:33:41.042588 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b90823-ad19-4790-a354-4b73085502c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7673a3b5c0324e07a9535ac70ed7a491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:33:41.043293 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b90823-ad19-4790-a354-4b73085502c3 service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:33:41.043774 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b90823-ad19-4790-a354-4b73085502c3 service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:33:41.043774 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b90823-ad19-4790-a354-4b73085502c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:33:41.053039 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b90823-ad19-4790-a354-4b73085502c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:33:41.054650 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b90823-ad19-4790-a354-4b73085502c3 service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:33:41.064266 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 761/1524] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:33:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:33:41.886130 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc70da0-b4da-4a23-9554-8adb624d99f8 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:33:41.886692 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc70da0-b4da-4a23-9554-8adb624d99f8 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:33:41.886692 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc70da0-b4da-4a23-9554-8adb624d99f8 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:33:41.888125 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 764/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 15 19:33:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:33:41.893252 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba822fa1-3a94-4a7c-a854-1bc84be3b8c0 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:33:41.893612 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba822fa1-3a94-4a7c-a854-1bc84be3b8c0 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:33:41.893925 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba822fa1-3a94-4a7c-a854-1bc84be3b8c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:33:41.912060 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-ba822fa1-3a94-4a7c-a854-1bc84be3b8c0 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:33:41.913254 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-ba822fa1-3a94-4a7c-a854-1bc84be3b8c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:33:41.921064 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 762/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 15 19:33:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:33:42.072900 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc70da0-b4da-4a23-9554-8adb624d99f8 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:33:42.086987 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc70da0-b4da-4a23-9554-8adb624d99f8 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:33:42.092419 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b77d0d-3eb0-41d5-87df-4e2d2cbd0bdf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7673a3b5c0324e07a9535ac70ed7a491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:33:42.093052 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b77d0d-3eb0-41d5-87df-4e2d2cbd0bdf service nova] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:33:42.093337 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b77d0d-3eb0-41d5-87df-4e2d2cbd0bdf service nova] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:33:42.093616 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b77d0d-3eb0-41d5-87df-4e2d2cbd0bdf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:33:42.101784 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b77d0d-3eb0-41d5-87df-4e2d2cbd0bdf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:33:42.103356 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b77d0d-3eb0-41d5-87df-4e2d2cbd0bdf service nova] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:33:42.112714 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 765/1527] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:33:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:33:45.940319 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d7d5e-3501-4ffb-831a-f598783d92bf None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:33:45.940873 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d7d5e-3501-4ffb-831a-f598783d92bf None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:33:45.940873 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d7d5e-3501-4ffb-831a-f598783d92bf None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:33:45.942827 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 763/1528] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 19:33:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:33:45.947971 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde400db-2cda-4e51-a540-22d182cf1d29 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:33:45.948409 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde400db-2cda-4e51-a540-22d182cf1d29 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:33:45.948596 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde400db-2cda-4e51-a540-22d182cf1d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:33:45.966189 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-dde400db-2cda-4e51-a540-22d182cf1d29 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:33:45.967388 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-dde400db-2cda-4e51-a540-22d182cf1d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:33:45.975224 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 766/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 19:33:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:33:46.012834 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d7d5e-3501-4ffb-831a-f598783d92bf None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:33:46.014530 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-fb7d7d5e-3501-4ffb-831a-f598783d92bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:33:46.092036 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d7d5e-3501-4ffb-831a-f598783d92bf None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:33:46.095564 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cd620c-c270-46a2-aff2-83c10a632779 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:33:46.096209 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cd620c-c270-46a2-aff2-83c10a632779 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:33:46.096492 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cd620c-c270-46a2-aff2-83c10a632779 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:33:46.096768 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cd620c-c270-46a2-aff2-83c10a632779 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:33:46.106878 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cd620c-c270-46a2-aff2-83c10a632779 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:33:46.108315 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cd620c-c270-46a2-aff2-83c10a632779 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:33:46.118159 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 764/1530] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:33:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:33:48.134501 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5e8f1a-9582-4c05-8c71-307840a149bb None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:33:48.135043 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5e8f1a-9582-4c05-8c71-307840a149bb None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:33:48.135043 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5e8f1a-9582-4c05-8c71-307840a149bb None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:33:48.136350 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 767/1531] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 19:33:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:33:48.141848 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d973c219-7c15-45af-8c0e-19dd2f95aaae None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:33:48.142245 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d973c219-7c15-45af-8c0e-19dd2f95aaae None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:33:48.142434 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d973c219-7c15-45af-8c0e-19dd2f95aaae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:33:48.161320 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-d973c219-7c15-45af-8c0e-19dd2f95aaae None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:33:48.162552 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d973c219-7c15-45af-8c0e-19dd2f95aaae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:33:48.170436 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 765/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 19:33:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:33:48.205327 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5e8f1a-9582-4c05-8c71-307840a149bb None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:33:48.216342 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5e8f1a-9582-4c05-8c71-307840a149bb None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:33:48.219797 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e37ad7-9261-40dd-812a-3aba350fd258 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:33:48.220418 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e37ad7-9261-40dd-812a-3aba350fd258 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:33:48.220696 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e37ad7-9261-40dd-812a-3aba350fd258 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:33:48.220964 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e37ad7-9261-40dd-812a-3aba350fd258 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:33:48.230386 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e37ad7-9261-40dd-812a-3aba350fd258 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:33:48.231790 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e37ad7-9261-40dd-812a-3aba350fd258 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:33:48.241017 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 768/1533] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:33:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:33:51.717960 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533e6c37-0ede-4b7a-81f3-5d04e7b259e6 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:33:51.718549 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533e6c37-0ede-4b7a-81f3-5d04e7b259e6 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:33:51.718657 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533e6c37-0ede-4b7a-81f3-5d04e7b259e6 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:33:51.720043 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 766/1534] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 19:33:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:33:51.725190 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ec426-c4d7-4a8e-9d88-35ba79e16322 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:33:51.725641 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ec426-c4d7-4a8e-9d88-35ba79e16322 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:33:51.725748 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ec426-c4d7-4a8e-9d88-35ba79e16322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:33:51.743656 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-397ec426-c4d7-4a8e-9d88-35ba79e16322 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:33:51.744851 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-397ec426-c4d7-4a8e-9d88-35ba79e16322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:33:51.752568 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 769/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 19:33:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:33:52.055443 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533e6c37-0ede-4b7a-81f3-5d04e7b259e6 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:33:52.057152 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-533e6c37-0ede-4b7a-81f3-5d04e7b259e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:33:52.121913 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:33:52.135975 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533e6c37-0ede-4b7a-81f3-5d04e7b259e6 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:33:52.139979 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d811251-e148-492a-b272-fedc673ee3e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86c2df3d378490ca7ea497a73c19d40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:33:52.140610 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d811251-e148-492a-b272-fedc673ee3e7 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:33:52.140895 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d811251-e148-492a-b272-fedc673ee3e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:33:52.141175 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d811251-e148-492a-b272-fedc673ee3e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:33:52.150704 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d811251-e148-492a-b272-fedc673ee3e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:33:52.152139 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d811251-e148-492a-b272-fedc673ee3e7 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:33:52.162176 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 767/1536] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:33:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:33:54.192359 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461cd074-d67c-4789-a35d-5656810445e8 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:33:54.192899 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461cd074-d67c-4789-a35d-5656810445e8 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:33:54.192899 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461cd074-d67c-4789-a35d-5656810445e8 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:33:54.194331 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 770/1537] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 19:33:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:33:54.200257 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb70bdd-d2ac-4cc5-b36d-a39292624b87 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:33:54.200715 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb70bdd-d2ac-4cc5-b36d-a39292624b87 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:33:54.200816 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb70bdd-d2ac-4cc5-b36d-a39292624b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:33:54.218651 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-afb70bdd-d2ac-4cc5-b36d-a39292624b87 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:33:54.219851 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-afb70bdd-d2ac-4cc5-b36d-a39292624b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:33:54.227856 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 768/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 19:33:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:33:54.549272 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461cd074-d67c-4789-a35d-5656810445e8 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:33:54.551524 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-461cd074-d67c-4789-a35d-5656810445e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:33:54.564266 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461cd074-d67c-4789-a35d-5656810445e8 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:33:54.567776 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666e43ab-39f6-4f49-9421-b1ce77ad0a5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86c2df3d378490ca7ea497a73c19d40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:33:54.568402 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e43ab-39f6-4f49-9421-b1ce77ad0a5a service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:33:54.568680 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e43ab-39f6-4f49-9421-b1ce77ad0a5a service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:33:54.568953 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e43ab-39f6-4f49-9421-b1ce77ad0a5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:33:54.577700 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666e43ab-39f6-4f49-9421-b1ce77ad0a5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:33:54.579129 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666e43ab-39f6-4f49-9421-b1ce77ad0a5a service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:33:54.588500 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 771/1539] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:33:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:34:04.214021 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=11 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:34:04.584382 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=33 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:35:25.500028 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238e0b12-fee2-4654-af53-8f477385e9bd None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:25.500726 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238e0b12-fee2-4654-af53-8f477385e9bd None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:25.500726 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238e0b12-fee2-4654-af53-8f477385e9bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:25.525044 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-238e0b12-fee2-4654-af53-8f477385e9bd None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:35:25.526283 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-238e0b12-fee2-4654-af53-8f477385e9bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:25.541208 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 769/1540] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:35:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:35:25.550439 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b66a1b-7f06-4c59-b28b-76a5c75ce0de None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:25.551433 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b66a1b-7f06-4c59-b28b-76a5c75ce0de None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:25.551988 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b66a1b-7f06-4c59-b28b-76a5c75ce0de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:25.584161 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-80b66a1b-7f06-4c59-b28b-76a5c75ce0de None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:35:25.585717 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-80b66a1b-7f06-4c59-b28b-76a5c75ce0de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:25.602964 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 772/1541] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:35:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:35:25.608300 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238e0b12-fee2-4654-af53-8f477385e9bd None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:25.625367 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238e0b12-fee2-4654-af53-8f477385e9bd None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:25.633560 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f0d3a4-2144-42fa-85b0-8985b75b422b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:25.634313 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f0d3a4-2144-42fa-85b0-8985b75b422b admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:25.634596 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f0d3a4-2144-42fa-85b0-8985b75b422b admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:25.634869 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f0d3a4-2144-42fa-85b0-8985b75b422b admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:25.635614 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f0d3a4-2144-42fa-85b0-8985b75b422b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:35:25.636502 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f0d3a4-2144-42fa-85b0-8985b75b422b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:25.638647 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f0d3a4-2144-42fa-85b0-8985b75b422b admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:25.659183 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 770/1542] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jul 15 19:35:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:35:25.667604 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b66a1b-7f06-4c59-b28b-76a5c75ce0de None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:25.684332 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b66a1b-7f06-4c59-b28b-76a5c75ce0de None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:25.700493 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ebdbf3-c9ba-456f-8b09-7b9435b9a920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:25.701541 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ebdbf3-c9ba-456f-8b09-7b9435b9a920 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:25.710558 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ebdbf3-c9ba-456f-8b09-7b9435b9a920 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:25.710558 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ebdbf3-c9ba-456f-8b09-7b9435b9a920 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:25.710558 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ebdbf3-c9ba-456f-8b09-7b9435b9a920 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:25.710558 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ebdbf3-c9ba-456f-8b09-7b9435b9a920 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:25.795400 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a2daf4-f04c-4070-b269-7315e7bf7e97 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:25.795690 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a2daf4-f04c-4070-b269-7315e7bf7e97 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:25.796213 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a2daf4-f04c-4070-b269-7315e7bf7e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:25.828262 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-d5a2daf4-f04c-4070-b269-7315e7bf7e97 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:35:25.829518 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d5a2daf4-f04c-4070-b269-7315e7bf7e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:25.842969 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 771/1543] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:35:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:35:25.851011 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff659480-b718-4d64-a093-f611bdc7bfeb None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:25.859687 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff659480-b718-4d64-a093-f611bdc7bfeb None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:25.859687 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff659480-b718-4d64-a093-f611bdc7bfeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:25.859925 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 773/1544] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jul 15 19:35:25 2022] POST /identity/v3/projects => generated 383 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:35:25.864096 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ebdbf3-c9ba-456f-8b09-7b9435b9a920 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:25.878231 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-ff659480-b718-4d64-a093-f611bdc7bfeb None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:35:25.879439 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-ff659480-b718-4d64-a093-f611bdc7bfeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:25.991328 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ebdbf3-c9ba-456f-8b09-7b9435b9a920 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:26.005704 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b757fd-5edb-4fb5-9d49-9c3f414178bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:26.006432 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b757fd-5edb-4fb5-9d49-9c3f414178bb admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:26.006718 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b757fd-5edb-4fb5-9d49-9c3f414178bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:26.006830 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 772/1545] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:35:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:35:26.007075 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b757fd-5edb-4fb5-9d49-9c3f414178bb admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:26.007708 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b757fd-5edb-4fb5-9d49-9c3f414178bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:26.008761 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: WARNING py.warnings [None req-71b757fd-5edb-4fb5-9d49-9c3f414178bb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 15 19:35:26.008761 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: warnings.warn(msg) Jul 15 19:35:26.008761 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: Jul 15 19:35:26.010041 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b757fd-5edb-4fb5-9d49-9c3f414178bb admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:26.011900 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff659480-b718-4d64-a093-f611bdc7bfeb None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:26.028390 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff659480-b718-4d64-a093-f611bdc7bfeb None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:26.047382 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2e00d6-d875-450d-b9df-2027acf86a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:26.048082 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e00d6-d875-450d-b9df-2027acf86a8c admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:26.048361 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e00d6-d875-450d-b9df-2027acf86a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:26.048639 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e00d6-d875-450d-b9df-2027acf86a8c admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:26.049657 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2e00d6-d875-450d-b9df-2027acf86a8c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:35:26.050007 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2e00d6-d875-450d-b9df-2027acf86a8c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:26.051626 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2e00d6-d875-450d-b9df-2027acf86a8c admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:26.073309 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 773/1546] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jul 15 19:35:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:35:26.079469 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f9c2e0-f36a-4709-a1c8-15d35a54241c None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:26.079829 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f9c2e0-f36a-4709-a1c8-15d35a54241c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:26.080351 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f9c2e0-f36a-4709-a1c8-15d35a54241c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:26.104994 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-c1f9c2e0-f36a-4709-a1c8-15d35a54241c None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:35:26.106164 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-c1f9c2e0-f36a-4709-a1c8-15d35a54241c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:26.122844 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 774/1547] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:35:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:35:26.123533 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f9c2e0-f36a-4709-a1c8-15d35a54241c None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:26.134241 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f9c2e0-f36a-4709-a1c8-15d35a54241c None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:26.146220 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 774/1548] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jul 15 19:35:25 2022] POST /identity/v3/users => generated 292 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:35:26.146220 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42be90ad-2973-4dfa-b16f-7da49132b14b None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:26.146220 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42be90ad-2973-4dfa-b16f-7da49132b14b None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:26.146220 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42be90ad-2973-4dfa-b16f-7da49132b14b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:26.146645 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2abbe1-7bd2-4f85-b001-e942466d33f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:26.146645 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2abbe1-7bd2-4f85-b001-e942466d33f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:26.146645 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2abbe1-7bd2-4f85-b001-e942466d33f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:26.146645 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2abbe1-7bd2-4f85-b001-e942466d33f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:26.147199 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2abbe1-7bd2-4f85-b001-e942466d33f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:26.148419 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: WARNING py.warnings [None req-1a2abbe1-7bd2-4f85-b001-e942466d33f3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 15 19:35:26.148419 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: warnings.warn(msg) Jul 15 19:35:26.148419 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: Jul 15 19:35:26.150056 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2abbe1-7bd2-4f85-b001-e942466d33f3 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:26.186216 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-42be90ad-2973-4dfa-b16f-7da49132b14b None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:35:26.187798 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-42be90ad-2973-4dfa-b16f-7da49132b14b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:26.202537 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 775/1549] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:35:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:35:26.205720 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42be90ad-2973-4dfa-b16f-7da49132b14b None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:26.220047 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42be90ad-2973-4dfa-b16f-7da49132b14b None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:26.227816 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e573ee79-ae7e-4a86-bd40-10abc409407c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:26.232849 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573ee79-ae7e-4a86-bd40-10abc409407c admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:26.232849 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573ee79-ae7e-4a86-bd40-10abc409407c admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:26.232849 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573ee79-ae7e-4a86-bd40-10abc409407c admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:26.232849 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e573ee79-ae7e-4a86-bd40-10abc409407c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:26.232849 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e573ee79-ae7e-4a86-bd40-10abc409407c admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:26.252047 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 776/1550] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:35:26 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:35:26.253225 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e573ee79-ae7e-4a86-bd40-10abc409407c admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:26.272254 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e573ee79-ae7e-4a86-bd40-10abc409407c admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:26.274981 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 775/1551] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jul 15 19:35:26 2022] POST /identity/v3/projects => generated 369 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:35:26.276442 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f4090b-c4cf-424e-8afd-acc01c1f0206 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:26.276737 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f4090b-c4cf-424e-8afd-acc01c1f0206 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:26.277013 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f4090b-c4cf-424e-8afd-acc01c1f0206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:26.301344 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-f5f4090b-c4cf-424e-8afd-acc01c1f0206 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:35:26.303053 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-f5f4090b-c4cf-424e-8afd-acc01c1f0206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:26.382591 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14d33d7-3920-424c-b253-58cea1a8f21b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:26.383572 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14d33d7-3920-424c-b253-58cea1a8f21b admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:26.384024 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14d33d7-3920-424c-b253-58cea1a8f21b admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:26.384471 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14d33d7-3920-424c-b253-58cea1a8f21b admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:26.385619 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14d33d7-3920-424c-b253-58cea1a8f21b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:35:26.386120 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14d33d7-3920-424c-b253-58cea1a8f21b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:26.387400 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: WARNING py.warnings [None req-e14d33d7-3920-424c-b253-58cea1a8f21b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 15 19:35:26.387400 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: warnings.warn(msg) Jul 15 19:35:26.387400 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: Jul 15 19:35:26.389060 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14d33d7-3920-424c-b253-58cea1a8f21b admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:26.390894 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 776/1552] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:35:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:35:26.392186 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f4090b-c4cf-424e-8afd-acc01c1f0206 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:26.403811 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f4090b-c4cf-424e-8afd-acc01c1f0206 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:26.416068 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 777/1553] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jul 15 19:35:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:35:26.416068 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81064ea-3fc1-49f2-a7fb-6ea496d1e322 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:26.416068 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81064ea-3fc1-49f2-a7fb-6ea496d1e322 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:26.416068 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81064ea-3fc1-49f2-a7fb-6ea496d1e322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:26.416487 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77f0906-a532-4fe8-b322-ab5318de6b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:26.416487 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77f0906-a532-4fe8-b322-ab5318de6b3f admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:26.416487 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77f0906-a532-4fe8-b322-ab5318de6b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:26.416487 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77f0906-a532-4fe8-b322-ab5318de6b3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:26.416871 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77f0906-a532-4fe8-b322-ab5318de6b3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:26.416871 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77f0906-a532-4fe8-b322-ab5318de6b3f admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:26.423516 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 777/1554] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:35:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:35:26.424345 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77f0906-a532-4fe8-b322-ab5318de6b3f admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:26.427787 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-e81064ea-3fc1-49f2-a7fb-6ea496d1e322 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:35:26.429021 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-e81064ea-3fc1-49f2-a7fb-6ea496d1e322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:26.434830 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77f0906-a532-4fe8-b322-ab5318de6b3f admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:26.437718 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 778/1555] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:35:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:35:26.439620 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d78a0b-5132-41ca-8946-6bea6fae0068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:26.440288 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d78a0b-5132-41ca-8946-6bea6fae0068 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:26.440570 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d78a0b-5132-41ca-8946-6bea6fae0068 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:26.440842 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d78a0b-5132-41ca-8946-6bea6fae0068 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:26.441523 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d78a0b-5132-41ca-8946-6bea6fae0068 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:26.443743 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d27bc25-870e-4471-b1f3-3686ef9e9d32 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:26.444059 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d78a0b-5132-41ca-8946-6bea6fae0068 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:26.444184 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d27bc25-870e-4471-b1f3-3686ef9e9d32 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:26.444353 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d27bc25-870e-4471-b1f3-3686ef9e9d32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:26.463775 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-2d27bc25-870e-4471-b1f3-3686ef9e9d32 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:35:26.465103 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-2d27bc25-870e-4471-b1f3-3686ef9e9d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:26.473253 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 779/1556] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:35:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:35:26.474528 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61073d32-d541-4a77-bbd7-09019e3df798 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:26.474815 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61073d32-d541-4a77-bbd7-09019e3df798 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:26.475094 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61073d32-d541-4a77-bbd7-09019e3df798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:26.493313 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-61073d32-d541-4a77-bbd7-09019e3df798 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:35:26.494621 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-61073d32-d541-4a77-bbd7-09019e3df798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:26.508011 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 780/1557] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:35:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:35:26.509957 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fede31b3-326a-4e29-80ce-c1077d676d99 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:26.509957 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fede31b3-326a-4e29-80ce-c1077d676d99 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:26.509957 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fede31b3-326a-4e29-80ce-c1077d676d99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:26.530548 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-fede31b3-326a-4e29-80ce-c1077d676d99 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:35:26.531781 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-fede31b3-326a-4e29-80ce-c1077d676d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:26.540270 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 781/1558] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:35:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:35:26.541548 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5dc897-a401-40a6-8c80-aa75292eeea6 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:26.541872 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5dc897-a401-40a6-8c80-aa75292eeea6 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:26.542149 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5dc897-a401-40a6-8c80-aa75292eeea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:26.548456 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:35:26.552978 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 778/1559] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jul 15 19:35:26 2022] POST /identity/v3/users => generated 285 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:35:26.552978 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d78a0b-5132-41ca-8946-6bea6fae0068 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:26.559736 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:35:26.561036 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-0c5dc897-a401-40a6-8c80-aa75292eeea6 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:35:26.562269 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-0c5dc897-a401-40a6-8c80-aa75292eeea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:26.563662 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d78a0b-5132-41ca-8946-6bea6fae0068 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:26.568491 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31f39ea-bc89-4f94-b9dd-725a12a9d7e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:26.569226 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31f39ea-bc89-4f94-b9dd-725a12a9d7e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:26.569579 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31f39ea-bc89-4f94-b9dd-725a12a9d7e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:26.569967 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31f39ea-bc89-4f94-b9dd-725a12a9d7e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:26.570961 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31f39ea-bc89-4f94-b9dd-725a12a9d7e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:26.571082 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 782/1560] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:35:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:35:26.571686 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5dc897-a401-40a6-8c80-aa75292eeea6 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:26.572840 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31f39ea-bc89-4f94-b9dd-725a12a9d7e2 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:26.584271 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5dc897-a401-40a6-8c80-aa75292eeea6 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:26.588906 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeee49f-d7f5-40db-b5af-15b1bc735b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:26.604612 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeee49f-d7f5-40db-b5af-15b1bc735b8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:26.604612 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeee49f-d7f5-40db-b5af-15b1bc735b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:26.604612 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeee49f-d7f5-40db-b5af-15b1bc735b8e admin admin] PATH_INFO: `/v3/projects/a84088d3693f448dadc1c58699966ed6/users/cd105befee3d452a87501a903496e459/roles/28763c6e70094116b3a4b2895895fa58` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:26.606738 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eeee49f-d7f5-40db-b5af-15b1bc735b8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a84088d3693f448dadc1c58699966ed6, user_id=cd105befee3d452a87501a903496e459, role_id=28763c6e70094116b3a4b2895895fa58)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:26.609396 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eeee49f-d7f5-40db-b5af-15b1bc735b8e admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:26.721091 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 779/1561] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jul 15 19:35:26 2022] POST /identity/v3/projects => generated 393 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:35:26.722349 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd74c4e-3506-4409-9666-5fccd2cd9d4d None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:26.722637 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd74c4e-3506-4409-9666-5fccd2cd9d4d None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:26.722911 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd74c4e-3506-4409-9666-5fccd2cd9d4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:26.740432 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-2dd74c4e-3506-4409-9666-5fccd2cd9d4d None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:35:26.741714 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-2dd74c4e-3506-4409-9666-5fccd2cd9d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:26.754088 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 783/1562] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:35:26 2022] PUT /identity/v3/projects/a84088d3693f448dadc1c58699966ed6/users/cd105befee3d452a87501a903496e459/roles/28763c6e70094116b3a4b2895895fa58 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:35:26.759140 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf96870b-cce1-4bec-aa06-2a1c97ad3c78 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:26.759140 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf96870b-cce1-4bec-aa06-2a1c97ad3c78 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:26.759140 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf96870b-cce1-4bec-aa06-2a1c97ad3c78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:26.776818 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-cf96870b-cce1-4bec-aa06-2a1c97ad3c78 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:35:26.778058 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-cf96870b-cce1-4bec-aa06-2a1c97ad3c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:26.808985 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 780/1563] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:35:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:35:26.808985 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4176c9-584d-4d7e-9aae-c66a6381a2cf None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:26.808985 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4176c9-584d-4d7e-9aae-c66a6381a2cf None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:26.808985 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4176c9-584d-4d7e-9aae-c66a6381a2cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:26.825977 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-ea4176c9-584d-4d7e-9aae-c66a6381a2cf None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:35:26.827174 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-ea4176c9-584d-4d7e-9aae-c66a6381a2cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:26.841938 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 784/1564] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:35:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:35:26.841938 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf96870b-cce1-4bec-aa06-2a1c97ad3c78 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:26.848174 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf96870b-cce1-4bec-aa06-2a1c97ad3c78 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:26.854324 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2a1f7c-b9da-447e-859c-5ca4f7f0936c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:26.855203 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2a1f7c-b9da-447e-859c-5ca4f7f0936c admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:26.866215 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2a1f7c-b9da-447e-859c-5ca4f7f0936c admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:26.866215 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2a1f7c-b9da-447e-859c-5ca4f7f0936c admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:26.866215 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2a1f7c-b9da-447e-859c-5ca4f7f0936c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:35:26.866215 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2a1f7c-b9da-447e-859c-5ca4f7f0936c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:26.866215 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2a1f7c-b9da-447e-859c-5ca4f7f0936c admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:26.870878 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 785/1565] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jul 15 19:35:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:35:26.871841 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2a1f7c-b9da-447e-859c-5ca4f7f0936c admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:26.887387 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 781/1566] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:35:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:35:26.889416 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2a1f7c-b9da-447e-859c-5ca4f7f0936c admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:26.889576 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5f19f0-9fe9-4922-b6ab-fa1b9fdb03ba None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:26.889826 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5f19f0-9fe9-4922-b6ab-fa1b9fdb03ba None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:26.890104 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5f19f0-9fe9-4922-b6ab-fa1b9fdb03ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:26.894721 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c2e089-81f7-4b44-8fed-73e30030ccf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:26.895667 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c2e089-81f7-4b44-8fed-73e30030ccf1 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:26.896474 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c2e089-81f7-4b44-8fed-73e30030ccf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:26.896759 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c2e089-81f7-4b44-8fed-73e30030ccf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:26.897491 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c2e089-81f7-4b44-8fed-73e30030ccf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:26.899135 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c2e089-81f7-4b44-8fed-73e30030ccf1 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:26.908428 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-0d5f19f0-9fe9-4922-b6ab-fa1b9fdb03ba None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:35:26.909623 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0d5f19f0-9fe9-4922-b6ab-fa1b9fdb03ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:26.909987 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 786/1567] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:35:26 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:35:26.913028 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c2e089-81f7-4b44-8fed-73e30030ccf1 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:26.919236 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 782/1568] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:35:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:35:26.920352 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c2e089-81f7-4b44-8fed-73e30030ccf1 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:26.920525 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cb83f4-7ed9-4c01-a1b8-b45c1cd6971a None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:26.920814 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cb83f4-7ed9-4c01-a1b8-b45c1cd6971a None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:26.921088 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cb83f4-7ed9-4c01-a1b8-b45c1cd6971a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:26.927402 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bdfca7-72fc-4222-8b77-ecbd51ee5563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:26.928055 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bdfca7-72fc-4222-8b77-ecbd51ee5563 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:26.928341 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bdfca7-72fc-4222-8b77-ecbd51ee5563 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:26.928611 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bdfca7-72fc-4222-8b77-ecbd51ee5563 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:26.929300 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bdfca7-72fc-4222-8b77-ecbd51ee5563 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:26.931247 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bdfca7-72fc-4222-8b77-ecbd51ee5563 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:26.947863 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-69cb83f4-7ed9-4c01-a1b8-b45c1cd6971a None None] MFA Rules not processed for user `cd105befee3d452a87501a903496e459`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:35:26.949399 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-69cb83f4-7ed9-4c01-a1b8-b45c1cd6971a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:27.011926 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 783/1569] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:35:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:35:27.012931 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cb83f4-7ed9-4c01-a1b8-b45c1cd6971a None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:27.024322 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cb83f4-7ed9-4c01-a1b8-b45c1cd6971a None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:27.030367 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93ecabc-c1d7-4682-b611-08b9e5f7e002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:27.035220 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 787/1570] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jul 15 19:35:26 2022] POST /identity/v3/users => generated 297 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:35:27.035873 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ecabc-c1d7-4682-b611-08b9e5f7e002 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:27.036108 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bdfca7-72fc-4222-8b77-ecbd51ee5563 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:27.036213 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ecabc-c1d7-4682-b611-08b9e5f7e002 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:27.036213 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ecabc-c1d7-4682-b611-08b9e5f7e002 admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:27.036213 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93ecabc-c1d7-4682-b611-08b9e5f7e002 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:35:27.036213 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93ecabc-c1d7-4682-b611-08b9e5f7e002 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:27.036213 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93ecabc-c1d7-4682-b611-08b9e5f7e002 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:27.036213 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:35:27.043615 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bdfca7-72fc-4222-8b77-ecbd51ee5563 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:27.056488 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 784/1571] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jul 15 19:35:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:35:27.056488 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93ecabc-c1d7-4682-b611-08b9e5f7e002 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:27.056770 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681e42cf-e9d8-420e-90d3-62a1fca29a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:27.057421 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e42cf-e9d8-420e-90d3-62a1fca29a64 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:27.057708 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e42cf-e9d8-420e-90d3-62a1fca29a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:27.058022 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e42cf-e9d8-420e-90d3-62a1fca29a64 admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:27.058761 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681e42cf-e9d8-420e-90d3-62a1fca29a64 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:35:27.059088 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681e42cf-e9d8-420e-90d3-62a1fca29a64 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:27.060838 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681e42cf-e9d8-420e-90d3-62a1fca29a64 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:27.066779 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93ecabc-c1d7-4682-b611-08b9e5f7e002 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:27.072126 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e30fab2-35ad-41bb-90a7-c3c5e9334c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:27.072779 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e30fab2-35ad-41bb-90a7-c3c5e9334c33 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:27.073062 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e30fab2-35ad-41bb-90a7-c3c5e9334c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:27.073339 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e30fab2-35ad-41bb-90a7-c3c5e9334c33 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:27.074194 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e30fab2-35ad-41bb-90a7-c3c5e9334c33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:27.074447 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 788/1572] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jul 15 19:35:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:35:27.075423 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681e42cf-e9d8-420e-90d3-62a1fca29a64 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:27.076494 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e30fab2-35ad-41bb-90a7-c3c5e9334c33 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:27.086583 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681e42cf-e9d8-420e-90d3-62a1fca29a64 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:27.091337 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5101fcf-d54a-45dd-a604-34cd8ead74d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:27.091994 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5101fcf-d54a-45dd-a604-34cd8ead74d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:27.092276 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5101fcf-d54a-45dd-a604-34cd8ead74d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:27.092555 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5101fcf-d54a-45dd-a604-34cd8ead74d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:27.093288 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5101fcf-d54a-45dd-a604-34cd8ead74d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:35:27.093597 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5101fcf-d54a-45dd-a604-34cd8ead74d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:27.095261 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5101fcf-d54a-45dd-a604-34cd8ead74d6 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:27.106982 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 789/1573] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jul 15 19:35:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:35:27.107780 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5101fcf-d54a-45dd-a604-34cd8ead74d6 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:27.117634 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5101fcf-d54a-45dd-a604-34cd8ead74d6 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:27.122216 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7d8260-c882-49c6-8656-8aff5790a74f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:27.122852 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7d8260-c882-49c6-8656-8aff5790a74f admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:27.123136 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7d8260-c882-49c6-8656-8aff5790a74f admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:27.123415 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7d8260-c882-49c6-8656-8aff5790a74f admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:27.124130 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7d8260-c882-49c6-8656-8aff5790a74f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:27.125688 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7d8260-c882-49c6-8656-8aff5790a74f admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:27.136693 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 790/1574] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:35:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:35:27.137729 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7d8260-c882-49c6-8656-8aff5790a74f admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:27.147088 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7d8260-c882-49c6-8656-8aff5790a74f admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:27.151796 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f76554-3973-412c-a25b-b06f4bf7d6c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:27.152448 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f76554-3973-412c-a25b-b06f4bf7d6c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:27.152732 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f76554-3973-412c-a25b-b06f4bf7d6c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:27.153009 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f76554-3973-412c-a25b-b06f4bf7d6c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:27.153717 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f76554-3973-412c-a25b-b06f4bf7d6c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:35:27.154063 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f76554-3973-412c-a25b-b06f4bf7d6c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:27.155633 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f76554-3973-412c-a25b-b06f4bf7d6c0 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:27.167613 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 791/1575] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jul 15 19:35:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:35:27.168431 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f76554-3973-412c-a25b-b06f4bf7d6c0 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:27.177647 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 785/1576] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jul 15 19:35:27 2022] POST /identity/v3/projects => generated 373 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:35:27.191246 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f76554-3973-412c-a25b-b06f4bf7d6c0 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:27.191405 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b31e5a5-dcb7-4cf1-8b95-898950b9960e None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:27.191405 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b31e5a5-dcb7-4cf1-8b95-898950b9960e None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:27.191405 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b31e5a5-dcb7-4cf1-8b95-898950b9960e None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:27.191405 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 786/1577] 10.222.0.33 () {54 vars in 961 bytes} [Fri Jul 15 19:35:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:35:27.191405 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b31e5a5-dcb7-4cf1-8b95-898950b9960e None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:27.221021 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:35:27.240281 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6aba7d-c9e2-4ed8-b1ab-c4418139fcbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:27.240954 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6aba7d-c9e2-4ed8-b1ab-c4418139fcbd admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:27.241239 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6aba7d-c9e2-4ed8-b1ab-c4418139fcbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:27.241518 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6aba7d-c9e2-4ed8-b1ab-c4418139fcbd admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:27.242285 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6aba7d-c9e2-4ed8-b1ab-c4418139fcbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:27.243887 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6aba7d-c9e2-4ed8-b1ab-c4418139fcbd admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:27.251552 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b31e5a5-dcb7-4cf1-8b95-898950b9960e None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:27.253705 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 792/1578] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:35:27 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:35:27.254628 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6aba7d-c9e2-4ed8-b1ab-c4418139fcbd admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:27.257334 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5eef88-3d91-4097-bb12-10c2beb079c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:27.258238 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5eef88-3d91-4097-bb12-10c2beb079c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:27.258520 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5eef88-3d91-4097-bb12-10c2beb079c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:27.258792 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5eef88-3d91-4097-bb12-10c2beb079c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:27.259480 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5eef88-3d91-4097-bb12-10c2beb079c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:27.261452 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5eef88-3d91-4097-bb12-10c2beb079c6 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:27.266889 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6aba7d-c9e2-4ed8-b1ab-c4418139fcbd admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:27.271472 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ecadd0-51bf-416f-9787-14982f0031f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:27.272112 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ecadd0-51bf-416f-9787-14982f0031f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:27.272392 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ecadd0-51bf-416f-9787-14982f0031f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:27.272663 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ecadd0-51bf-416f-9787-14982f0031f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:27.273337 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ecadd0-51bf-416f-9787-14982f0031f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:27.275236 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ecadd0-51bf-416f-9787-14982f0031f4 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:27.411761 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 793/1579] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jul 15 19:35:27 2022] POST /identity/v3/projects => generated 379 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:35:27.412833 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ecadd0-51bf-416f-9787-14982f0031f4 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:27.424149 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 787/1580] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jul 15 19:35:27 2022] POST /identity/v3/projects => generated 371 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:35:27.425176 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5eef88-3d91-4097-bb12-10c2beb079c6 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:27.479344 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ecadd0-51bf-416f-9787-14982f0031f4 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:27.489146 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5eef88-3d91-4097-bb12-10c2beb079c6 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:27.494436 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14e9d6a-50b1-4719-8254-6675a1a72c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:27.495409 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14e9d6a-50b1-4719-8254-6675a1a72c69 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:27.495697 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14e9d6a-50b1-4719-8254-6675a1a72c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:27.495970 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14e9d6a-50b1-4719-8254-6675a1a72c69 admin admin] PATH_INFO: `/v3/projects/47ec617f4907433cb56564f63f23c749/users/03c59537825243ae9a38863718c9ebdd/roles/28763c6e70094116b3a4b2895895fa58` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:27.510476 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14e9d6a-50b1-4719-8254-6675a1a72c69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47ec617f4907433cb56564f63f23c749, user_id=03c59537825243ae9a38863718c9ebdd, role_id=28763c6e70094116b3a4b2895895fa58)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:27.511611 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: WARNING py.warnings [None req-f14e9d6a-50b1-4719-8254-6675a1a72c69 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 15 19:35:27.511611 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: warnings.warn(msg) Jul 15 19:35:27.511611 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: Jul 15 19:35:27.513855 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14e9d6a-50b1-4719-8254-6675a1a72c69 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:27.537087 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88d3fe4-b395-4ebe-a69c-625bec7d985c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:27.537831 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88d3fe4-b395-4ebe-a69c-625bec7d985c admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:27.538156 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88d3fe4-b395-4ebe-a69c-625bec7d985c admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:27.538431 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88d3fe4-b395-4ebe-a69c-625bec7d985c admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:27.539112 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88d3fe4-b395-4ebe-a69c-625bec7d985c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:27.540995 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88d3fe4-b395-4ebe-a69c-625bec7d985c admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:27.606513 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 788/1581] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:35:27 2022] PUT /identity/v3/projects/47ec617f4907433cb56564f63f23c749/users/03c59537825243ae9a38863718c9ebdd/roles/28763c6e70094116b3a4b2895895fa58 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:35:27.607324 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14e9d6a-50b1-4719-8254-6675a1a72c69 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:27.650834 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:35:27.653097 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 794/1582] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jul 15 19:35:27 2022] POST /identity/v3/projects => generated 375 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:35:27.657624 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11ee4cc-21f8-46f7-b3e7-f6520b246712 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:27.657925 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11ee4cc-21f8-46f7-b3e7-f6520b246712 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:27.658214 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11ee4cc-21f8-46f7-b3e7-f6520b246712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:27.672156 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14e9d6a-50b1-4719-8254-6675a1a72c69 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:27.700765 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-d11ee4cc-21f8-46f7-b3e7-f6520b246712 None None] MFA Rules not processed for user `7673a3b5c0324e07a9535ac70ed7a491`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:35:27.702013 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-d11ee4cc-21f8-46f7-b3e7-f6520b246712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:27.731974 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ba21cc-c479-49f3-89a5-82d15547a47b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:27.732802 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba21cc-c479-49f3-89a5-82d15547a47b admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:27.733083 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba21cc-c479-49f3-89a5-82d15547a47b admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:27.733358 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba21cc-c479-49f3-89a5-82d15547a47b admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:27.734093 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ba21cc-c479-49f3-89a5-82d15547a47b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:27.734224 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:35:27.736147 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ba21cc-c479-49f3-89a5-82d15547a47b admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:27.772840 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 795/1583] 10.222.0.33 () {58 vars in 1058 bytes} [Fri Jul 15 19:35:27 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:35:27.773679 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11ee4cc-21f8-46f7-b3e7-f6520b246712 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:27.783702 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11ee4cc-21f8-46f7-b3e7-f6520b246712 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:27.789055 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61784c6-b9fa-4fa2-89a5-cc84d88460b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:27.789649 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61784c6-b9fa-4fa2-89a5-cc84d88460b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:27.789962 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61784c6-b9fa-4fa2-89a5-cc84d88460b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:27.790244 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61784c6-b9fa-4fa2-89a5-cc84d88460b3 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:27.790934 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61784c6-b9fa-4fa2-89a5-cc84d88460b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:27.792847 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61784c6-b9fa-4fa2-89a5-cc84d88460b3 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:27.846620 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 789/1584] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jul 15 19:35:27 2022] POST /identity/v3/projects => generated 389 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:35:27.847608 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ba21cc-c479-49f3-89a5-82d15547a47b admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:27.909289 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ba21cc-c479-49f3-89a5-82d15547a47b admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:27.913808 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 796/1585] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jul 15 19:35:27 2022] POST /identity/v3/users => generated 287 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:35:27.919189 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61784c6-b9fa-4fa2-89a5-cc84d88460b3 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:27.919311 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3787796-e5ce-471a-a989-1b117c37f081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:27.919311 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3787796-e5ce-471a-a989-1b117c37f081 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:27.919311 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3787796-e5ce-471a-a989-1b117c37f081 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:27.919311 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3787796-e5ce-471a-a989-1b117c37f081 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:27.919696 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3787796-e5ce-471a-a989-1b117c37f081 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:27.921337 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3787796-e5ce-471a-a989-1b117c37f081 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:27.925109 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61784c6-b9fa-4fa2-89a5-cc84d88460b3 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:27.930030 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdd10e4-69df-47c3-afa0-92ca67c51d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:27.930691 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdd10e4-69df-47c3-afa0-92ca67c51d76 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:27.930992 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdd10e4-69df-47c3-afa0-92ca67c51d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:27.931270 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdd10e4-69df-47c3-afa0-92ca67c51d76 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:27.931953 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdd10e4-69df-47c3-afa0-92ca67c51d76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:27.932163 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 790/1586] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:35:27 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:35:27.932830 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3787796-e5ce-471a-a989-1b117c37f081 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:27.934263 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdd10e4-69df-47c3-afa0-92ca67c51d76 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:27.945463 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3787796-e5ce-471a-a989-1b117c37f081 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:27.950195 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49a6281-8d02-4052-a100-2575b9886e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:27.950851 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49a6281-8d02-4052-a100-2575b9886e6f admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:27.951132 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49a6281-8d02-4052-a100-2575b9886e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:27.951405 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49a6281-8d02-4052-a100-2575b9886e6f admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:27.952090 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49a6281-8d02-4052-a100-2575b9886e6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:27.954059 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49a6281-8d02-4052-a100-2575b9886e6f admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:28.044639 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 797/1587] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jul 15 19:35:27 2022] POST /identity/v3/users => generated 290 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:35:28.045988 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fceb0c6-cfe1-4662-a3a6-e2369c2c91d6 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:28.046281 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fceb0c6-cfe1-4662-a3a6-e2369c2c91d6 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:28.046566 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fceb0c6-cfe1-4662-a3a6-e2369c2c91d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:28.058345 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 791/1588] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jul 15 19:35:27 2022] POST /identity/v3/users => generated 286 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:35:28.058345 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49a6281-8d02-4052-a100-2575b9886e6f admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:28.066185 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49a6281-8d02-4052-a100-2575b9886e6f admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:28.071341 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27afe03-3652-4eaa-b9b1-c07719cf4304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:28.071991 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27afe03-3652-4eaa-b9b1-c07719cf4304 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:28.072273 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27afe03-3652-4eaa-b9b1-c07719cf4304 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:28.072552 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27afe03-3652-4eaa-b9b1-c07719cf4304 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:28.073363 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27afe03-3652-4eaa-b9b1-c07719cf4304 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:28.074504 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-7fceb0c6-cfe1-4662-a3a6-e2369c2c91d6 None None] MFA Rules not processed for user `03c59537825243ae9a38863718c9ebdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:35:28.075338 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27afe03-3652-4eaa-b9b1-c07719cf4304 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:28.075868 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-7fceb0c6-cfe1-4662-a3a6-e2369c2c91d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:28.140632 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 798/1589] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:35:28 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:35:28.141560 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fceb0c6-cfe1-4662-a3a6-e2369c2c91d6 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:28.187830 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 792/1590] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jul 15 19:35:28 2022] POST /identity/v3/users => generated 288 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:35:28.188746 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27afe03-3652-4eaa-b9b1-c07719cf4304 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:28.199153 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27afe03-3652-4eaa-b9b1-c07719cf4304 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:28.203858 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cbd972-0a46-4c4a-a18d-1f0eef3f431e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:28.204531 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cbd972-0a46-4c4a-a18d-1f0eef3f431e admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:28.204812 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cbd972-0a46-4c4a-a18d-1f0eef3f431e admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:28.205159 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cbd972-0a46-4c4a-a18d-1f0eef3f431e admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:28.205824 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cbd972-0a46-4c4a-a18d-1f0eef3f431e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:28.207841 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cbd972-0a46-4c4a-a18d-1f0eef3f431e admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:28.215817 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fceb0c6-cfe1-4662-a3a6-e2369c2c91d6 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:28.221620 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52453a1-f9a9-4293-b72f-b98fe54e18f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7673a3b5c0324e07a9535ac70ed7a491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:28.222324 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52453a1-f9a9-4293-b72f-b98fe54e18f3 service nova] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:28.222607 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52453a1-f9a9-4293-b72f-b98fe54e18f3 service nova] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:28.222884 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52453a1-f9a9-4293-b72f-b98fe54e18f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:28.236945 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:35:28.279698 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52453a1-f9a9-4293-b72f-b98fe54e18f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:28.283169 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 793/1591] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jul 15 19:35:28 2022] POST /identity/v3/users => generated 295 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:35:28.283169 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cbd972-0a46-4c4a-a18d-1f0eef3f431e admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:28.283392 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52453a1-f9a9-4293-b72f-b98fe54e18f3 service nova] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:28.290832 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 799/1592] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:35:28 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:35:28.290990 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cbd972-0a46-4c4a-a18d-1f0eef3f431e admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:28.291539 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52453a1-f9a9-4293-b72f-b98fe54e18f3 service nova] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:28.295930 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601f4b79-375a-4005-9d17-857556172b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:28.296777 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601f4b79-375a-4005-9d17-857556172b9b admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:28.297276 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601f4b79-375a-4005-9d17-857556172b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:28.297713 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601f4b79-375a-4005-9d17-857556172b9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:28.298630 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601f4b79-375a-4005-9d17-857556172b9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:28.300237 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601f4b79-375a-4005-9d17-857556172b9b admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:28.301526 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52453a1-f9a9-4293-b72f-b98fe54e18f3 service nova] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:28.306742 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b4b192-8e48-43ab-8fa5-1b32fb54ef75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:28.308651 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b4b192-8e48-43ab-8fa5-1b32fb54ef75 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:28.308651 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b4b192-8e48-43ab-8fa5-1b32fb54ef75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:28.308651 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b4b192-8e48-43ab-8fa5-1b32fb54ef75 admin admin] PATH_INFO: `/v3/projects/6793b8b5e7e24faf9b5e3feb466adc08/users/8a654f77867341a1adf8db7f36ddc653/roles/28763c6e70094116b3a4b2895895fa58` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:28.312617 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 794/1593] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:35:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:35:28.313639 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601f4b79-375a-4005-9d17-857556172b9b admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:28.326040 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:35:28.326489 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b4b192-8e48-43ab-8fa5-1b32fb54ef75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6793b8b5e7e24faf9b5e3feb466adc08, user_id=8a654f77867341a1adf8db7f36ddc653, role_id=28763c6e70094116b3a4b2895895fa58)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:28.329265 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b4b192-8e48-43ab-8fa5-1b32fb54ef75 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:28.330587 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601f4b79-375a-4005-9d17-857556172b9b admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:28.336535 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee9cc57-8f4a-455a-9c73-fa90453da25e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:28.337327 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee9cc57-8f4a-455a-9c73-fa90453da25e admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:28.337632 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee9cc57-8f4a-455a-9c73-fa90453da25e admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:28.337945 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee9cc57-8f4a-455a-9c73-fa90453da25e admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:28.338678 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee9cc57-8f4a-455a-9c73-fa90453da25e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:28.340309 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee9cc57-8f4a-455a-9c73-fa90453da25e admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:28.351093 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 795/1594] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:35:28 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:35:28.351093 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee9cc57-8f4a-455a-9c73-fa90453da25e admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:28.361598 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee9cc57-8f4a-455a-9c73-fa90453da25e admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:28.366873 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89091a1-ede1-452b-8d04-106c0700230b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:28.367545 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89091a1-ede1-452b-8d04-106c0700230b admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:28.367832 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89091a1-ede1-452b-8d04-106c0700230b admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:28.368109 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89091a1-ede1-452b-8d04-106c0700230b admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:28.368830 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89091a1-ede1-452b-8d04-106c0700230b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:28.370497 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89091a1-ede1-452b-8d04-106c0700230b admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:28.380646 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 796/1595] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:35:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:35:28.382046 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89091a1-ede1-452b-8d04-106c0700230b admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:28.391544 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89091a1-ede1-452b-8d04-106c0700230b admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:28.396154 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36c4e1e-d448-46dd-9921-067849a21d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:28.397305 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36c4e1e-d448-46dd-9921-067849a21d8c admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:28.397587 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36c4e1e-d448-46dd-9921-067849a21d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:28.397896 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36c4e1e-d448-46dd-9921-067849a21d8c admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:28.398597 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36c4e1e-d448-46dd-9921-067849a21d8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:28.400484 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36c4e1e-d448-46dd-9921-067849a21d8c admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:28.457086 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 800/1596] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:35:28 2022] PUT /identity/v3/projects/6793b8b5e7e24faf9b5e3feb466adc08/users/8a654f77867341a1adf8db7f36ddc653/roles/28763c6e70094116b3a4b2895895fa58 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:35:28.461658 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b4b192-8e48-43ab-8fa5-1b32fb54ef75 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:28.524588 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 797/1597] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jul 15 19:35:28 2022] POST /identity/v3/projects => generated 367 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:35:28.526034 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36c4e1e-d448-46dd-9921-067849a21d8c admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:28.529157 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b4b192-8e48-43ab-8fa5-1b32fb54ef75 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:28.588047 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd535a44-c104-445b-86d8-0daa01e6232a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:28.588874 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd535a44-c104-445b-86d8-0daa01e6232a admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:28.589259 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd535a44-c104-445b-86d8-0daa01e6232a admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:28.589732 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd535a44-c104-445b-86d8-0daa01e6232a admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:28.590602 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd535a44-c104-445b-86d8-0daa01e6232a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:28.592427 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd535a44-c104-445b-86d8-0daa01e6232a admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:28.593792 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36c4e1e-d448-46dd-9921-067849a21d8c admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:28.598601 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876eb3d8-203e-4a98-bb29-02262e8c0285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:28.599374 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876eb3d8-203e-4a98-bb29-02262e8c0285 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:28.599764 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876eb3d8-203e-4a98-bb29-02262e8c0285 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:28.600038 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876eb3d8-203e-4a98-bb29-02262e8c0285 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:28.600756 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876eb3d8-203e-4a98-bb29-02262e8c0285 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:28.602373 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876eb3d8-203e-4a98-bb29-02262e8c0285 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:28.602922 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 801/1598] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:35:28 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:35:28.603789 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd535a44-c104-445b-86d8-0daa01e6232a admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:28.613169 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 798/1599] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:35:28 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:35:28.614028 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876eb3d8-203e-4a98-bb29-02262e8c0285 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:28.614633 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd535a44-c104-445b-86d8-0daa01e6232a admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:28.620125 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5b1e77-d498-4831-b78c-27a524440db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:28.620785 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5b1e77-d498-4831-b78c-27a524440db7 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:28.621068 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5b1e77-d498-4831-b78c-27a524440db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:28.621346 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5b1e77-d498-4831-b78c-27a524440db7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:28.622098 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5b1e77-d498-4831-b78c-27a524440db7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:28.623695 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5b1e77-d498-4831-b78c-27a524440db7 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:28.624696 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876eb3d8-203e-4a98-bb29-02262e8c0285 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:28.629269 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6825a2-c4e9-4cd6-aece-9b2e29003372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:28.629951 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6825a2-c4e9-4cd6-aece-9b2e29003372 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:28.630230 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6825a2-c4e9-4cd6-aece-9b2e29003372 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:28.630504 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6825a2-c4e9-4cd6-aece-9b2e29003372 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:28.631787 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6825a2-c4e9-4cd6-aece-9b2e29003372 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:28.633506 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6825a2-c4e9-4cd6-aece-9b2e29003372 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:28.635203 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 802/1600] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:35:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:35:28.636036 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5b1e77-d498-4831-b78c-27a524440db7 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:28.647566 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 799/1601] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:35:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:35:28.647566 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82887793-e52f-4568-9af7-57bb2c113f56 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:28.647566 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82887793-e52f-4568-9af7-57bb2c113f56 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:28.648045 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5b1e77-d498-4831-b78c-27a524440db7 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:28.648152 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82887793-e52f-4568-9af7-57bb2c113f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:28.651816 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd42c07f-5f36-4ead-b3e7-80117c26bfca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:28.652513 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd42c07f-5f36-4ead-b3e7-80117c26bfca admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:28.652795 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd42c07f-5f36-4ead-b3e7-80117c26bfca admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:28.653070 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd42c07f-5f36-4ead-b3e7-80117c26bfca admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:28.653845 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd42c07f-5f36-4ead-b3e7-80117c26bfca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:28.655494 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd42c07f-5f36-4ead-b3e7-80117c26bfca admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:28.669838 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 803/1602] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:35:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:35:28.670686 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd42c07f-5f36-4ead-b3e7-80117c26bfca admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:28.671753 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-82887793-e52f-4568-9af7-57bb2c113f56 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:35:28.673103 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-82887793-e52f-4568-9af7-57bb2c113f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:28.685757 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 800/1603] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:35:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:35:28.686695 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd42c07f-5f36-4ead-b3e7-80117c26bfca admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:28.690120 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92797a6-be9c-4761-a002-7d47a706ed7f None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:28.692707 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92797a6-be9c-4761-a002-7d47a706ed7f None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:28.693006 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92797a6-be9c-4761-a002-7d47a706ed7f None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:28.694490 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1c90d3-84ab-4939-96af-d9d2f3e62b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:28.695275 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c90d3-84ab-4939-96af-d9d2f3e62b15 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:28.695847 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c90d3-84ab-4939-96af-d9d2f3e62b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:28.696266 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c90d3-84ab-4939-96af-d9d2f3e62b15 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:28.696965 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 801/1604] 10.222.0.33 () {54 vars in 912 bytes} [Fri Jul 15 19:35:28 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:35:28.697596 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92797a6-be9c-4761-a002-7d47a706ed7f None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:28.698199 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1c90d3-84ab-4939-96af-d9d2f3e62b15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:28.701352 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1c90d3-84ab-4939-96af-d9d2f3e62b15 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:28.712693 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92797a6-be9c-4761-a002-7d47a706ed7f None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:28.712896 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:35:28.718549 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375172ab-c5cf-4821-9f50-2afa79f44bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:28.725660 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375172ab-c5cf-4821-9f50-2afa79f44bf4 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:28.725660 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375172ab-c5cf-4821-9f50-2afa79f44bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:28.725660 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375172ab-c5cf-4821-9f50-2afa79f44bf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:28.725660 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375172ab-c5cf-4821-9f50-2afa79f44bf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:28.725660 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375172ab-c5cf-4821-9f50-2afa79f44bf4 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:28.734300 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 802/1605] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:35:28 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:35:28.735175 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375172ab-c5cf-4821-9f50-2afa79f44bf4 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:28.745490 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375172ab-c5cf-4821-9f50-2afa79f44bf4 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:28.750045 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787ef616-3da2-4a9c-b525-0160b638ff60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:28.750669 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787ef616-3da2-4a9c-b525-0160b638ff60 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:28.750948 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787ef616-3da2-4a9c-b525-0160b638ff60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:28.751226 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787ef616-3da2-4a9c-b525-0160b638ff60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:28.751943 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787ef616-3da2-4a9c-b525-0160b638ff60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:28.753481 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787ef616-3da2-4a9c-b525-0160b638ff60 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:28.764249 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 803/1606] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:35:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:35:28.764939 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787ef616-3da2-4a9c-b525-0160b638ff60 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:28.774177 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787ef616-3da2-4a9c-b525-0160b638ff60 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:28.778694 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fcadfd-9f43-4284-b621-05a1afc880ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:28.779546 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fcadfd-9f43-4284-b621-05a1afc880ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:28.779827 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fcadfd-9f43-4284-b621-05a1afc880ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:28.781850 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fcadfd-9f43-4284-b621-05a1afc880ee admin admin] PATH_INFO: `/v3/projects/ce489d0b15524c3ebb161ac10ec1a7c3/users/5dc28ae2cadf44e596f41eee5ae3bdc0/roles/28763c6e70094116b3a4b2895895fa58` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:28.796402 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fcadfd-9f43-4284-b621-05a1afc880ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce489d0b15524c3ebb161ac10ec1a7c3, user_id=5dc28ae2cadf44e596f41eee5ae3bdc0, role_id=28763c6e70094116b3a4b2895895fa58)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:28.799049 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fcadfd-9f43-4284-b621-05a1afc880ee admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:28.811355 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 804/1607] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jul 15 19:35:28 2022] POST /identity/v3/users => generated 290 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:35:28.812151 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1c90d3-84ab-4939-96af-d9d2f3e62b15 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:28.821757 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1c90d3-84ab-4939-96af-d9d2f3e62b15 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:28.826494 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b1bb16-3808-446d-b0fa-6350403ade51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:28.827364 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b1bb16-3808-446d-b0fa-6350403ade51 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:28.827642 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b1bb16-3808-446d-b0fa-6350403ade51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:28.828011 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b1bb16-3808-446d-b0fa-6350403ade51 admin admin] PATH_INFO: `/v3/projects/7797a303c4d24e988742fcbebbfa6baa/users/3d2ee510f0a14305ad13f85d2e0b3a49/roles/28763c6e70094116b3a4b2895895fa58` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:28.842351 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b1bb16-3808-446d-b0fa-6350403ade51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7797a303c4d24e988742fcbebbfa6baa, user_id=3d2ee510f0a14305ad13f85d2e0b3a49, role_id=28763c6e70094116b3a4b2895895fa58)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:28.845053 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b1bb16-3808-446d-b0fa-6350403ade51 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:28.897039 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 804/1608] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:35:28 2022] PUT /identity/v3/projects/ce489d0b15524c3ebb161ac10ec1a7c3/users/5dc28ae2cadf44e596f41eee5ae3bdc0/roles/28763c6e70094116b3a4b2895895fa58 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:35:28.897880 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fcadfd-9f43-4284-b621-05a1afc880ee admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:28.925467 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 805/1609] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:35:28 2022] PUT /identity/v3/projects/7797a303c4d24e988742fcbebbfa6baa/users/3d2ee510f0a14305ad13f85d2e0b3a49/roles/28763c6e70094116b3a4b2895895fa58 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:35:28.926461 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b1bb16-3808-446d-b0fa-6350403ade51 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:28.928989 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-40b1bb16-3808-446d-b0fa-6350403ade51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:28.954119 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:35:28.964458 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fcadfd-9f43-4284-b621-05a1afc880ee admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:28.987272 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b1bb16-3808-446d-b0fa-6350403ade51 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:28.991857 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8f5473-e1e4-4224-ad69-98d6899c0282 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:28.992689 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f5473-e1e4-4224-ad69-98d6899c0282 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:28.998185 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f5473-e1e4-4224-ad69-98d6899c0282 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:28.998185 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f5473-e1e4-4224-ad69-98d6899c0282 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:29.022241 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91d10ab-330a-4c3d-a7df-f1e98bf823ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:29.023385 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91d10ab-330a-4c3d-a7df-f1e98bf823ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:29.023722 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91d10ab-330a-4c3d-a7df-f1e98bf823ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:29.023998 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91d10ab-330a-4c3d-a7df-f1e98bf823ac admin admin] PATH_INFO: `/v3/projects/16b8b4b0c5fe4769b61df425b5585ea3/users/72ff9ead5e494aa188af528d5dff62c9/roles/28763c6e70094116b3a4b2895895fa58` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:29.045696 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91d10ab-330a-4c3d-a7df-f1e98bf823ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=16b8b4b0c5fe4769b61df425b5585ea3, user_id=72ff9ead5e494aa188af528d5dff62c9, role_id=28763c6e70094116b3a4b2895895fa58)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:29.048368 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91d10ab-330a-4c3d-a7df-f1e98bf823ac admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:29.063438 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8f5473-e1e4-4224-ad69-98d6899c0282 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:29.064801 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8f5473-e1e4-4224-ad69-98d6899c0282 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:29.074448 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 806/1610] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:35:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:35:29.075704 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e1197c-81cb-4d5c-b7c5-274dabeaedee None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:29.075994 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e1197c-81cb-4d5c-b7c5-274dabeaedee None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:29.076272 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e1197c-81cb-4d5c-b7c5-274dabeaedee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:29.093147 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-d8e1197c-81cb-4d5c-b7c5-274dabeaedee None None] MFA Rules not processed for user `7673a3b5c0324e07a9535ac70ed7a491`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:35:29.095680 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-d8e1197c-81cb-4d5c-b7c5-274dabeaedee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:29.121032 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 805/1611] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:35:28 2022] PUT /identity/v3/projects/16b8b4b0c5fe4769b61df425b5585ea3/users/72ff9ead5e494aa188af528d5dff62c9/roles/28763c6e70094116b3a4b2895895fa58 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:35:29.122088 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91d10ab-330a-4c3d-a7df-f1e98bf823ac admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:29.164040 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 807/1612] 10.222.0.33 () {58 vars in 1009 bytes} [Fri Jul 15 19:35:29 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:35:29.165005 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e1197c-81cb-4d5c-b7c5-274dabeaedee None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:29.176222 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e1197c-81cb-4d5c-b7c5-274dabeaedee None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:29.187059 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:35:29.192863 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91d10ab-330a-4c3d-a7df-f1e98bf823ac admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:29.199326 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fe39df-b749-41c8-b8a9-0c6809a6a11d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:29.200214 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fe39df-b749-41c8-b8a9-0c6809a6a11d admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:29.200491 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fe39df-b749-41c8-b8a9-0c6809a6a11d admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:29.200764 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fe39df-b749-41c8-b8a9-0c6809a6a11d admin admin] PATH_INFO: `/v3/projects/e18f974d83ad4c68a5a3b8bdae3bc4c7/users/eb2902b42e3449ba95b73ad5f85fcfdc/roles/28763c6e70094116b3a4b2895895fa58` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:29.219091 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fe39df-b749-41c8-b8a9-0c6809a6a11d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e18f974d83ad4c68a5a3b8bdae3bc4c7, user_id=eb2902b42e3449ba95b73ad5f85fcfdc, role_id=28763c6e70094116b3a4b2895895fa58)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:29.221784 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fe39df-b749-41c8-b8a9-0c6809a6a11d admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:29.240474 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3973f2ff-e285-4bd2-9062-25d15ed66aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:29.241374 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3973f2ff-e285-4bd2-9062-25d15ed66aac admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:29.241654 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3973f2ff-e285-4bd2-9062-25d15ed66aac admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:29.241958 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3973f2ff-e285-4bd2-9062-25d15ed66aac admin admin] PATH_INFO: `/v3/projects/70b48f0cb9ad47d2826f7ef2cbfa55e5/users/6193fdc75def489e88eda70e78ec6f46/roles/28763c6e70094116b3a4b2895895fa58` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:29.256883 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3973f2ff-e285-4bd2-9062-25d15ed66aac admin admin] RBAC: Authorizing `identity:create_grant(project_id=70b48f0cb9ad47d2826f7ef2cbfa55e5, user_id=6193fdc75def489e88eda70e78ec6f46, role_id=28763c6e70094116b3a4b2895895fa58)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:29.259771 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3973f2ff-e285-4bd2-9062-25d15ed66aac admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:29.291978 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 806/1613] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:35:29 2022] PUT /identity/v3/projects/e18f974d83ad4c68a5a3b8bdae3bc4c7/users/eb2902b42e3449ba95b73ad5f85fcfdc/roles/28763c6e70094116b3a4b2895895fa58 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:35:29.292813 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fe39df-b749-41c8-b8a9-0c6809a6a11d admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:29.354959 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fe39df-b749-41c8-b8a9-0c6809a6a11d admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:29.359517 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fa5d6a-e32f-4c74-af72-cd27de5b3e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:29.360179 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fa5d6a-e32f-4c74-af72-cd27de5b3e1d admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:29.360479 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fa5d6a-e32f-4c74-af72-cd27de5b3e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:29.360749 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fa5d6a-e32f-4c74-af72-cd27de5b3e1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:29.361464 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fa5d6a-e32f-4c74-af72-cd27de5b3e1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:29.363108 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fa5d6a-e32f-4c74-af72-cd27de5b3e1d admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:29.372850 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 808/1614] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:35:29 2022] PUT /identity/v3/projects/70b48f0cb9ad47d2826f7ef2cbfa55e5/users/6193fdc75def489e88eda70e78ec6f46/roles/28763c6e70094116b3a4b2895895fa58 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:35:29.376806 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 807/1615] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:35:29 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:35:29.376806 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05add7c6-fee0-4900-a845-66b9eeed7116 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:29.376806 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05add7c6-fee0-4900-a845-66b9eeed7116 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:29.376806 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05add7c6-fee0-4900-a845-66b9eeed7116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:29.378242 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd973bc-f807-46c7-b69c-153bb2d99ecb None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:29.378531 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd973bc-f807-46c7-b69c-153bb2d99ecb None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:29.378814 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd973bc-f807-46c7-b69c-153bb2d99ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:29.381788 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:35:29.394548 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-05add7c6-fee0-4900-a845-66b9eeed7116 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:35:29.395759 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-05add7c6-fee0-4900-a845-66b9eeed7116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:29.412354 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-5cd973bc-f807-46c7-b69c-153bb2d99ecb None None] MFA Rules not processed for user `5dc28ae2cadf44e596f41eee5ae3bdc0`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:35:29.413782 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-5cd973bc-f807-46c7-b69c-153bb2d99ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:29.473051 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 808/1616] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:35:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:35:29.474833 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfefe98-94ae-402e-94f6-600d7764ff42 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:29.475104 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfefe98-94ae-402e-94f6-600d7764ff42 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:29.475843 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfefe98-94ae-402e-94f6-600d7764ff42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:29.500628 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 809/1617] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:35:29 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:35:29.500628 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd973bc-f807-46c7-b69c-153bb2d99ecb None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:29.516237 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-8dfefe98-94ae-402e-94f6-600d7764ff42 None None] MFA Rules not processed for user `72ff9ead5e494aa188af528d5dff62c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:35:29.518057 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-8dfefe98-94ae-402e-94f6-600d7764ff42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:29.564865 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd973bc-f807-46c7-b69c-153bb2d99ecb None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:29.569218 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e434c5-dabb-4811-b724-2a65123ff2a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86c2df3d378490ca7ea497a73c19d40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:29.569789 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e434c5-dabb-4811-b724-2a65123ff2a3 service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:29.570750 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e434c5-dabb-4811-b724-2a65123ff2a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:29.571037 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e434c5-dabb-4811-b724-2a65123ff2a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:29.592396 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 809/1618] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:35:29 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:35:29.593819 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf5c73a-5c3c-43dc-a195-4f88bec62018 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:29.594183 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf5c73a-5c3c-43dc-a195-4f88bec62018 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:29.594530 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf5c73a-5c3c-43dc-a195-4f88bec62018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:29.615834 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-ccf5c73a-5c3c-43dc-a195-4f88bec62018 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:35:29.619263 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:35:29.631382 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-ccf5c73a-5c3c-43dc-a195-4f88bec62018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:29.649737 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 810/1619] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:35:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:35:29.650926 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf5c73a-5c3c-43dc-a195-4f88bec62018 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:29.657214 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e434c5-dabb-4811-b724-2a65123ff2a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:29.658696 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e434c5-dabb-4811-b724-2a65123ff2a3 service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:29.661548 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf5c73a-5c3c-43dc-a195-4f88bec62018 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:29.668209 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f351226-689a-4b0a-b1e6-4fc3c7ce876f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:29.669272 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f351226-689a-4b0a-b1e6-4fc3c7ce876f admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:29.669822 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f351226-689a-4b0a-b1e6-4fc3c7ce876f admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:29.670283 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f351226-689a-4b0a-b1e6-4fc3c7ce876f admin admin] PATH_INFO: `/v3/projects/8e18003e470a44cd94f7d2ee9ddcb07b/users/81d102b0d25e4550b37fad2789132f51/roles/4318f86ba9f74cd4a0bef1c311276657` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:29.671351 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 810/1620] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:35:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:35:29.685838 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5662cf98-ac86-4af0-adbe-2aa881b82ad6 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:29.686305 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5662cf98-ac86-4af0-adbe-2aa881b82ad6 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:29.686663 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5662cf98-ac86-4af0-adbe-2aa881b82ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:29.694739 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f351226-689a-4b0a-b1e6-4fc3c7ce876f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e18003e470a44cd94f7d2ee9ddcb07b, user_id=81d102b0d25e4550b37fad2789132f51, role_id=4318f86ba9f74cd4a0bef1c311276657)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:29.698394 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f351226-689a-4b0a-b1e6-4fc3c7ce876f admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:29.706795 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-5662cf98-ac86-4af0-adbe-2aa881b82ad6 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:35:29.708094 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-5662cf98-ac86-4af0-adbe-2aa881b82ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:29.717086 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 811/1621] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:35:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:35:29.717819 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5662cf98-ac86-4af0-adbe-2aa881b82ad6 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:29.795831 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5662cf98-ac86-4af0-adbe-2aa881b82ad6 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:29.799597 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a35ec2e-901e-487f-97ff-26c9aa0e2f1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:29.800537 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a35ec2e-901e-487f-97ff-26c9aa0e2f1a service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:29.800820 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a35ec2e-901e-487f-97ff-26c9aa0e2f1a service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:29.801087 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a35ec2e-901e-487f-97ff-26c9aa0e2f1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:29.813233 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a35ec2e-901e-487f-97ff-26c9aa0e2f1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:29.813506 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 811/1622] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:35:29 2022] PUT /identity/v3/projects/8e18003e470a44cd94f7d2ee9ddcb07b/users/81d102b0d25e4550b37fad2789132f51/roles/4318f86ba9f74cd4a0bef1c311276657 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:35:29.814674 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f351226-689a-4b0a-b1e6-4fc3c7ce876f admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:29.815199 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a35ec2e-901e-487f-97ff-26c9aa0e2f1a service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:29.830216 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 812/1623] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:35:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:35:29.831619 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d753f9-dd7e-4fc3-99bb-9a061c9ac3d8 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:29.831909 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d753f9-dd7e-4fc3-99bb-9a061c9ac3d8 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:29.832189 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d753f9-dd7e-4fc3-99bb-9a061c9ac3d8 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:29.833703 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 813/1624] 10.222.0.33 () {54 vars in 961 bytes} [Fri Jul 15 19:35:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:35:29.835503 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d753f9-dd7e-4fc3-99bb-9a061c9ac3d8 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:29.899461 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d753f9-dd7e-4fc3-99bb-9a061c9ac3d8 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:29.903959 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c14f749-b985-4661-b82c-7932ff107d68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:29.904613 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c14f749-b985-4661-b82c-7932ff107d68 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:29.904893 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c14f749-b985-4661-b82c-7932ff107d68 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:29.905165 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c14f749-b985-4661-b82c-7932ff107d68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:29.913261 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f351226-689a-4b0a-b1e6-4fc3c7ce876f admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:29.917195 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c14f749-b985-4661-b82c-7932ff107d68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:29.919666 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eee784-ff08-47eb-a122-96ac5c7d3883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:29.919837 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c14f749-b985-4661-b82c-7932ff107d68 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:29.920827 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eee784-ff08-47eb-a122-96ac5c7d3883 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:29.921345 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eee784-ff08-47eb-a122-96ac5c7d3883 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:29.921870 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eee784-ff08-47eb-a122-96ac5c7d3883 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:29.922676 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eee784-ff08-47eb-a122-96ac5c7d3883 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:29.925155 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eee784-ff08-47eb-a122-96ac5c7d3883 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:29.931062 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:35:29.933866 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 814/1625] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:35:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:35:29.934906 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c14f749-b985-4661-b82c-7932ff107d68 service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:29.955197 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c14f749-b985-4661-b82c-7932ff107d68 service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:29.960486 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b886c234-d726-4b95-ae6a-9b8f14aa9d2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:29.961033 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b886c234-d726-4b95-ae6a-9b8f14aa9d2e service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:29.961417 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b886c234-d726-4b95-ae6a-9b8f14aa9d2e service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:29.961818 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b886c234-d726-4b95-ae6a-9b8f14aa9d2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:29.972294 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b886c234-d726-4b95-ae6a-9b8f14aa9d2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:29.974100 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b886c234-d726-4b95-ae6a-9b8f14aa9d2e service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:29.986085 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 815/1626] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:35:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:35:29.987059 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b886c234-d726-4b95-ae6a-9b8f14aa9d2e service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:29.997952 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b886c234-d726-4b95-ae6a-9b8f14aa9d2e service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:30.004390 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ed7c33-dd04-45fd-8ba0-607c103d8add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:30.005263 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ed7c33-dd04-45fd-8ba0-607c103d8add admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:30.005614 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ed7c33-dd04-45fd-8ba0-607c103d8add admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:30.005969 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ed7c33-dd04-45fd-8ba0-607c103d8add admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:30.006806 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ed7c33-dd04-45fd-8ba0-607c103d8add admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:30.008517 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ed7c33-dd04-45fd-8ba0-607c103d8add admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:30.027448 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 812/1627] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jul 15 19:35:29 2022] POST /identity/v3/projects => generated 371 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:35:30.031333 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 816/1628] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:35:29 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:35:30.031508 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb7bcf4-aa28-4433-88ab-bea1a349296c None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:30.031609 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ed7c33-dd04-45fd-8ba0-607c103d8add admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:30.031711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb7bcf4-aa28-4433-88ab-bea1a349296c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:30.031711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb7bcf4-aa28-4433-88ab-bea1a349296c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:30.042471 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ed7c33-dd04-45fd-8ba0-607c103d8add admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:30.065353 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-6fb7bcf4-aa28-4433-88ab-bea1a349296c None None] MFA Rules not processed for user `7673a3b5c0324e07a9535ac70ed7a491`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:35:30.067276 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-6fb7bcf4-aa28-4433-88ab-bea1a349296c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:30.096621 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:35:30.113642 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688cbee0-dcd9-4a1d-a027-f64daa0a96dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:30.114576 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688cbee0-dcd9-4a1d-a027-f64daa0a96dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:30.114916 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688cbee0-dcd9-4a1d-a027-f64daa0a96dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:30.115221 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688cbee0-dcd9-4a1d-a027-f64daa0a96dc admin admin] PATH_INFO: `/v3/projects/8e18003e470a44cd94f7d2ee9ddcb07b/users/81d102b0d25e4550b37fad2789132f51/roles/4318f86ba9f74cd4a0bef1c311276657` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:30.117208 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688cbee0-dcd9-4a1d-a027-f64daa0a96dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e18003e470a44cd94f7d2ee9ddcb07b, user_id=81d102b0d25e4550b37fad2789132f51, role_id=4318f86ba9f74cd4a0bef1c311276657)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:30.119926 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688cbee0-dcd9-4a1d-a027-f64daa0a96dc admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:30.138848 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 817/1629] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:35:30 2022] PUT /identity/v3/projects/8e18003e470a44cd94f7d2ee9ddcb07b/users/81d102b0d25e4550b37fad2789132f51/roles/4318f86ba9f74cd4a0bef1c311276657 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:35:30.138848 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688cbee0-dcd9-4a1d-a027-f64daa0a96dc admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:30.154315 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 813/1630] 10.222.0.33 () {58 vars in 1058 bytes} [Fri Jul 15 19:35:30 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:35:30.156498 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1342bb-6d31-4073-9b12-23659b7a0dc7 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:30.159735 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1342bb-6d31-4073-9b12-23659b7a0dc7 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:30.159735 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1342bb-6d31-4073-9b12-23659b7a0dc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:30.194172 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-1b1342bb-6d31-4073-9b12-23659b7a0dc7 None None] MFA Rules not processed for user `81d102b0d25e4550b37fad2789132f51`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:35:30.195512 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-1b1342bb-6d31-4073-9b12-23659b7a0dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:30.206807 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688cbee0-dcd9-4a1d-a027-f64daa0a96dc admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:30.213194 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266dfd85-e9c0-4033-818e-0f4bf22c2465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:30.214009 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266dfd85-e9c0-4033-818e-0f4bf22c2465 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:30.214295 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266dfd85-e9c0-4033-818e-0f4bf22c2465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:30.214570 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266dfd85-e9c0-4033-818e-0f4bf22c2465 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:30.215349 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266dfd85-e9c0-4033-818e-0f4bf22c2465 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:30.217370 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266dfd85-e9c0-4033-818e-0f4bf22c2465 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:30.305161 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 814/1631] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:35:30 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:35:30.306176 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1342bb-6d31-4073-9b12-23659b7a0dc7 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:30.355564 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 818/1632] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jul 15 19:35:30 2022] POST /identity/v3/users => generated 292 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:35:30.356912 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266dfd85-e9c0-4033-818e-0f4bf22c2465 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:30.414126 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:35:30.422044 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1342bb-6d31-4073-9b12-23659b7a0dc7 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:30.427853 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eef63d-e3bc-4f93-9cdd-8e2bea5dd46e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7673a3b5c0324e07a9535ac70ed7a491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:30.428631 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eef63d-e3bc-4f93-9cdd-8e2bea5dd46e service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:30.429014 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eef63d-e3bc-4f93-9cdd-8e2bea5dd46e service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:30.429410 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eef63d-e3bc-4f93-9cdd-8e2bea5dd46e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:30.434560 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266dfd85-e9c0-4033-818e-0f4bf22c2465 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:30.439741 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aadc8b-ad71-4786-8226-a7c428683c13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86c2df3d378490ca7ea497a73c19d40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:30.440657 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aadc8b-ad71-4786-8226-a7c428683c13 service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:30.441192 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aadc8b-ad71-4786-8226-a7c428683c13 service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:30.441568 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aadc8b-ad71-4786-8226-a7c428683c13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:30.453186 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68aadc8b-ad71-4786-8226-a7c428683c13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:30.455022 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68aadc8b-ad71-4786-8226-a7c428683c13 service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:30.466947 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 819/1633] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:35:30 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:35:30.467693 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aadc8b-ad71-4786-8226-a7c428683c13 service cinder] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:30.480833 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aadc8b-ad71-4786-8226-a7c428683c13 service cinder] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:30.487652 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669d8949-fb62-47c7-b8c4-3aeebd456efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:30.488416 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d8949-fb62-47c7-b8c4-3aeebd456efa admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:30.488964 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d8949-fb62-47c7-b8c4-3aeebd456efa admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:30.489089 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d8949-fb62-47c7-b8c4-3aeebd456efa admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:30.490540 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669d8949-fb62-47c7-b8c4-3aeebd456efa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:30.492414 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669d8949-fb62-47c7-b8c4-3aeebd456efa admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:30.507785 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 820/1634] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:35:30 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:35:30.511755 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669d8949-fb62-47c7-b8c4-3aeebd456efa admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:30.513124 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eef63d-e3bc-4f93-9cdd-8e2bea5dd46e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:30.515208 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eef63d-e3bc-4f93-9cdd-8e2bea5dd46e service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:30.526642 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 815/1635] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:35:30 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:35:30.529899 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669d8949-fb62-47c7-b8c4-3aeebd456efa admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:30.535802 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afdd8a1-7fad-49ea-b777-778f18e39c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:30.537630 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afdd8a1-7fad-49ea-b777-778f18e39c71 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:30.537630 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afdd8a1-7fad-49ea-b777-778f18e39c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:30.537630 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afdd8a1-7fad-49ea-b777-778f18e39c71 admin admin] PATH_INFO: `/v3/projects/fa570668a1b144ebb5dd90965d2d45aa/users/d28e0716521042688a4f697005069925/roles/4318f86ba9f74cd4a0bef1c311276657` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:30.571151 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afdd8a1-7fad-49ea-b777-778f18e39c71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa570668a1b144ebb5dd90965d2d45aa, user_id=d28e0716521042688a4f697005069925, role_id=4318f86ba9f74cd4a0bef1c311276657)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:30.576045 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afdd8a1-7fad-49ea-b777-778f18e39c71 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:30.713645 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 821/1636] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:35:30 2022] PUT /identity/v3/projects/fa570668a1b144ebb5dd90965d2d45aa/users/d28e0716521042688a4f697005069925/roles/4318f86ba9f74cd4a0bef1c311276657 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:35:30.718410 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eef63d-e3bc-4f93-9cdd-8e2bea5dd46e service nova] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:30.734152 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eef63d-e3bc-4f93-9cdd-8e2bea5dd46e service nova] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:30.798309 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d238c35-760d-4b68-b227-e56114331ce5 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:30.798674 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d238c35-760d-4b68-b227-e56114331ce5 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:30.798874 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d238c35-760d-4b68-b227-e56114331ce5 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:30.800257 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 822/1637] 10.222.0.33 () {54 vars in 912 bytes} [Fri Jul 15 19:35:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 19:35:30.806437 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec73bfa6-cf45-4c47-863b-5934c861713d None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:30.807419 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec73bfa6-cf45-4c47-863b-5934c861713d None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:30.809119 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec73bfa6-cf45-4c47-863b-5934c861713d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:30.820447 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ad41b4-8c6b-4def-bcc4-595400b222bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:30.821261 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ad41b4-8c6b-4def-bcc4-595400b222bf admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:30.821663 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ad41b4-8c6b-4def-bcc4-595400b222bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:30.822041 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ad41b4-8c6b-4def-bcc4-595400b222bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:30.822853 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ad41b4-8c6b-4def-bcc4-595400b222bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:30.824565 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ad41b4-8c6b-4def-bcc4-595400b222bf admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:30.853464 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-ec73bfa6-cf45-4c47-863b-5934c861713d None None] MFA Rules not processed for user `7673a3b5c0324e07a9535ac70ed7a491`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:35:30.855343 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-ec73bfa6-cf45-4c47-863b-5934c861713d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:30.861464 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 816/1638] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:35:30 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:35:30.864373 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ad41b4-8c6b-4def-bcc4-595400b222bf admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:30.903709 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ad41b4-8c6b-4def-bcc4-595400b222bf admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:30.927490 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477d3483-97ad-4f20-9821-195b488957e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:30.928942 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477d3483-97ad-4f20-9821-195b488957e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:30.929225 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477d3483-97ad-4f20-9821-195b488957e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:30.929616 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477d3483-97ad-4f20-9821-195b488957e9 admin admin] PATH_INFO: `/v3/projects/fa570668a1b144ebb5dd90965d2d45aa/users/d28e0716521042688a4f697005069925/roles/4318f86ba9f74cd4a0bef1c311276657` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:30.936246 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477d3483-97ad-4f20-9821-195b488957e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa570668a1b144ebb5dd90965d2d45aa, user_id=d28e0716521042688a4f697005069925, role_id=4318f86ba9f74cd4a0bef1c311276657)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:30.945511 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477d3483-97ad-4f20-9821-195b488957e9 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:30.959173 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:35:30.978215 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 817/1639] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:35:30 2022] PUT /identity/v3/projects/fa570668a1b144ebb5dd90965d2d45aa/users/d28e0716521042688a4f697005069925/roles/4318f86ba9f74cd4a0bef1c311276657 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:35:30.983554 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c7a679-3b00-4dbe-846f-35ce2522bdcd None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:30.983973 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c7a679-3b00-4dbe-846f-35ce2522bdcd None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:30.984271 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c7a679-3b00-4dbe-846f-35ce2522bdcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:31.076334 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-e6c7a679-3b00-4dbe-846f-35ce2522bdcd None None] MFA Rules not processed for user `d28e0716521042688a4f697005069925`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:35:31.077568 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e6c7a679-3b00-4dbe-846f-35ce2522bdcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:31.087601 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 823/1640] 10.222.0.33 () {58 vars in 1009 bytes} [Fri Jul 15 19:35:30 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:35:31.198326 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec73bfa6-cf45-4c47-863b-5934c861713d None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:31.304551 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 818/1641] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:35:30 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:35:31.366478 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec73bfa6-cf45-4c47-863b-5934c861713d None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:31.468023 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c7a679-3b00-4dbe-846f-35ce2522bdcd None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:31.473723 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b1578a-bf4c-462a-a62f-c4054d240b82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86c2df3d378490ca7ea497a73c19d40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:31.475915 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b1578a-bf4c-462a-a62f-c4054d240b82 service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:31.476524 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b1578a-bf4c-462a-a62f-c4054d240b82 service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:31.477377 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b1578a-bf4c-462a-a62f-c4054d240b82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:31.479841 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c7a679-3b00-4dbe-846f-35ce2522bdcd None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:31.485747 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a3903c-06f3-42a8-9142-7ee982003acd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86c2df3d378490ca7ea497a73c19d40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:31.486468 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a3903c-06f3-42a8-9142-7ee982003acd service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:31.486754 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a3903c-06f3-42a8-9142-7ee982003acd service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:31.487034 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a3903c-06f3-42a8-9142-7ee982003acd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:31.498232 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:35:31.502119 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a3903c-06f3-42a8-9142-7ee982003acd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:31.503767 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a3903c-06f3-42a8-9142-7ee982003acd service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:31.516129 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 819/1642] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:35:31 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:35:31.565105 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b1578a-bf4c-462a-a62f-c4054d240b82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:31.567347 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b1578a-bf4c-462a-a62f-c4054d240b82 service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:31.579420 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 824/1643] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:35:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 19:35:32.157884 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a3903c-06f3-42a8-9142-7ee982003acd service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:32.167068 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a3903c-06f3-42a8-9142-7ee982003acd service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:32.179342 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e920cf-712e-4521-9879-809697adb6cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86c2df3d378490ca7ea497a73c19d40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:32.180030 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e920cf-712e-4521-9879-809697adb6cc service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:32.180309 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e920cf-712e-4521-9879-809697adb6cc service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:32.180775 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e920cf-712e-4521-9879-809697adb6cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:32.244023 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e920cf-712e-4521-9879-809697adb6cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:32.245531 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e920cf-712e-4521-9879-809697adb6cc service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:32.255221 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 820/1644] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:35:32 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:35:37.991128 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b1578a-bf4c-462a-a62f-c4054d240b82 service cinder] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:38.072257 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:35:38.130052 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b1578a-bf4c-462a-a62f-c4054d240b82 service cinder] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:38.135107 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062b2a6d-fba7-415b-ae09-0bfa1bc462aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:38.136252 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062b2a6d-fba7-415b-ae09-0bfa1bc462aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:38.136669 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062b2a6d-fba7-415b-ae09-0bfa1bc462aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:38.137073 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062b2a6d-fba7-415b-ae09-0bfa1bc462aa admin admin] PATH_INFO: `/v3/users/03c59537825243ae9a38863718c9ebdd` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:38.138873 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062b2a6d-fba7-415b-ae09-0bfa1bc462aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=03c59537825243ae9a38863718c9ebdd)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:38.148013 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062b2a6d-fba7-415b-ae09-0bfa1bc462aa admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:38.401603 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-062b2a6d-fba7-415b-ae09-0bfa1bc462aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03c59537825243ae9a38863718c9ebdd'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:35:38.446128 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-062b2a6d-fba7-415b-ae09-0bfa1bc462aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03c59537825243ae9a38863718c9ebdd'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:35:38.542253 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-062b2a6d-fba7-415b-ae09-0bfa1bc462aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03c59537825243ae9a38863718c9ebdd'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:35:38.542937 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-062b2a6d-fba7-415b-ae09-0bfa1bc462aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03c59537825243ae9a38863718c9ebdd'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:35:38.543568 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-062b2a6d-fba7-415b-ae09-0bfa1bc462aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03c59537825243ae9a38863718c9ebdd'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:35:38.582606 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 825/1645] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jul 15 19:35:37 2022] DELETE /identity/v3/users/03c59537825243ae9a38863718c9ebdd => generated 0 bytes in 593 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:35:38.588932 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19fb671-2d94-46df-8760-cef61d0b1cb0 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:38.589485 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19fb671-2d94-46df-8760-cef61d0b1cb0 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:38.590381 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19fb671-2d94-46df-8760-cef61d0b1cb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:38.627142 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-d19fb671-2d94-46df-8760-cef61d0b1cb0 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:35:38.633557 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d19fb671-2d94-46df-8760-cef61d0b1cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:38.792706 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 821/1646] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:35:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:35:38.807955 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062b2a6d-fba7-415b-ae09-0bfa1bc462aa admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:38.809631 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-062b2a6d-fba7-415b-ae09-0bfa1bc462aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:38.918177 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062b2a6d-fba7-415b-ae09-0bfa1bc462aa admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:38.927528 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dca8dc-833c-4d2c-8d23-10555d190649 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:38.928209 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dca8dc-833c-4d2c-8d23-10555d190649 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:38.928491 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dca8dc-833c-4d2c-8d23-10555d190649 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:38.928810 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dca8dc-833c-4d2c-8d23-10555d190649 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:38.947654 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dca8dc-833c-4d2c-8d23-10555d190649 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:38.951458 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dca8dc-833c-4d2c-8d23-10555d190649 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:38.972752 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 826/1647] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:35:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:35:40.967413 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19fb671-2d94-46df-8760-cef61d0b1cb0 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:40.968892 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d19fb671-2d94-46df-8760-cef61d0b1cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:41.004763 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19fb671-2d94-46df-8760-cef61d0b1cb0 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:41.019683 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8271cc8-8f9c-4ee5-af86-3e90922f0c56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:41.023412 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8271cc8-8f9c-4ee5-af86-3e90922f0c56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:41.023701 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8271cc8-8f9c-4ee5-af86-3e90922f0c56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:41.025777 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8271cc8-8f9c-4ee5-af86-3e90922f0c56 admin admin] PATH_INFO: `/v3/projects/47ec617f4907433cb56564f63f23c749` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:41.027875 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8271cc8-8f9c-4ee5-af86-3e90922f0c56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47ec617f4907433cb56564f63f23c749)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:41.030406 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8271cc8-8f9c-4ee5-af86-3e90922f0c56 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:41.263974 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-c8271cc8-8f9c-4ee5-af86-3e90922f0c56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47ec617f4907433cb56564f63f23c749', 'request_id': 'req-c8271cc8-8f9c-4ee5-af86-3e90922f0c56'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:35:41.366573 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-c8271cc8-8f9c-4ee5-af86-3e90922f0c56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47ec617f4907433cb56564f63f23c749', 'request_id': 'req-c8271cc8-8f9c-4ee5-af86-3e90922f0c56'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:35:41.377379 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-c8271cc8-8f9c-4ee5-af86-3e90922f0c56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47ec617f4907433cb56564f63f23c749', 'request_id': 'req-c8271cc8-8f9c-4ee5-af86-3e90922f0c56'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:35:41.398883 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-c8271cc8-8f9c-4ee5-af86-3e90922f0c56 admin admin] The token cache is being invalidate because project 47ec617f4907433cb56564f63f23c749 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:35:41.399181 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-c8271cc8-8f9c-4ee5-af86-3e90922f0c56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:35:41.399907 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-c8271cc8-8f9c-4ee5-af86-3e90922f0c56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:35:41.402463 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 822/1648] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jul 15 19:35:40 2022] DELETE /identity/v3/projects/47ec617f4907433cb56564f63f23c749 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:35:41.414024 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dca8dc-833c-4d2c-8d23-10555d190649 service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:41.420316 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78203d38-e4bc-4e73-8e35-9005f38dc023 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:41.420760 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78203d38-e4bc-4e73-8e35-9005f38dc023 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:41.421416 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78203d38-e4bc-4e73-8e35-9005f38dc023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:41.429514 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dca8dc-833c-4d2c-8d23-10555d190649 service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:41.506230 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-78203d38-e4bc-4e73-8e35-9005f38dc023 None None] MFA Rules not processed for user `6193fdc75def489e88eda70e78ec6f46`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:35:41.507303 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-78203d38-e4bc-4e73-8e35-9005f38dc023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:41.525476 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:35:41.550090 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:35:41.576428 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688cde3c-a43f-4e55-a144-89769d7aae33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:41.577883 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688cde3c-a43f-4e55-a144-89769d7aae33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:41.578440 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688cde3c-a43f-4e55-a144-89769d7aae33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:41.579169 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688cde3c-a43f-4e55-a144-89769d7aae33 admin admin] PATH_INFO: `/v3/users/81d102b0d25e4550b37fad2789132f51` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:41.581844 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688cde3c-a43f-4e55-a144-89769d7aae33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81d102b0d25e4550b37fad2789132f51)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:41.597255 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688cde3c-a43f-4e55-a144-89769d7aae33 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:41.674707 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 823/1649] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:35:41 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:35:41.690094 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78203d38-e4bc-4e73-8e35-9005f38dc023 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:41.695056 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-78203d38-e4bc-4e73-8e35-9005f38dc023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:41.827637 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78203d38-e4bc-4e73-8e35-9005f38dc023 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:41.843643 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef781c2-7546-4903-9be2-ab85f48b6c26 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7673a3b5c0324e07a9535ac70ed7a491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:41.845942 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef781c2-7546-4903-9be2-ab85f48b6c26 service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:41.847132 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef781c2-7546-4903-9be2-ab85f48b6c26 service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:41.847966 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef781c2-7546-4903-9be2-ab85f48b6c26 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:41.856596 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-688cde3c-a43f-4e55-a144-89769d7aae33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81d102b0d25e4550b37fad2789132f51'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:35:41.868369 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef781c2-7546-4903-9be2-ab85f48b6c26 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:41.871584 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef781c2-7546-4903-9be2-ab85f48b6c26 service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:41.890093 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-688cde3c-a43f-4e55-a144-89769d7aae33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81d102b0d25e4550b37fad2789132f51'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:35:41.904357 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 824/1650] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:35:41 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:35:42.017530 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-688cde3c-a43f-4e55-a144-89769d7aae33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81d102b0d25e4550b37fad2789132f51'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:35:42.018551 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-688cde3c-a43f-4e55-a144-89769d7aae33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81d102b0d25e4550b37fad2789132f51'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:35:42.019809 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-688cde3c-a43f-4e55-a144-89769d7aae33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81d102b0d25e4550b37fad2789132f51'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:35:42.057112 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 827/1651] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jul 15 19:35:41 2022] DELETE /identity/v3/users/81d102b0d25e4550b37fad2789132f51 => generated 0 bytes in 643 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:35:42.395446 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7030b23-1c5d-4384-9797-71982e3c6db4 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:42.396032 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7030b23-1c5d-4384-9797-71982e3c6db4 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:42.396536 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7030b23-1c5d-4384-9797-71982e3c6db4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:42.489441 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-f7030b23-1c5d-4384-9797-71982e3c6db4 None None] MFA Rules not processed for user `8a654f77867341a1adf8db7f36ddc653`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:35:42.491443 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-f7030b23-1c5d-4384-9797-71982e3c6db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:42.700970 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 825/1652] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:35:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:35:42.706402 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab577c1-5787-42c4-98d4-6a8a8626cfb9 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:42.706402 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab577c1-5787-42c4-98d4-6a8a8626cfb9 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:42.706402 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab577c1-5787-42c4-98d4-6a8a8626cfb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:42.749372 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-eab577c1-5787-42c4-98d4-6a8a8626cfb9 None None] MFA Rules not processed for user `8a654f77867341a1adf8db7f36ddc653`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:35:42.752740 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-eab577c1-5787-42c4-98d4-6a8a8626cfb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:42.784869 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 828/1653] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:35:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:35:42.799991 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7030b23-1c5d-4384-9797-71982e3c6db4 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:42.801452 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-f7030b23-1c5d-4384-9797-71982e3c6db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:42.944536 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7030b23-1c5d-4384-9797-71982e3c6db4 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:42.954702 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b759a6c1-c49c-4979-9198-3ebbfd8c9ad6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7673a3b5c0324e07a9535ac70ed7a491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:42.955395 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b759a6c1-c49c-4979-9198-3ebbfd8c9ad6 service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:42.956790 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b759a6c1-c49c-4979-9198-3ebbfd8c9ad6 service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:42.957081 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b759a6c1-c49c-4979-9198-3ebbfd8c9ad6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:42.970846 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab577c1-5787-42c4-98d4-6a8a8626cfb9 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:42.972543 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-eab577c1-5787-42c4-98d4-6a8a8626cfb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:42.993066 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b759a6c1-c49c-4979-9198-3ebbfd8c9ad6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:42.994688 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b759a6c1-c49c-4979-9198-3ebbfd8c9ad6 service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:43.025654 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 826/1654] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:35:42 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:35:43.108281 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab577c1-5787-42c4-98d4-6a8a8626cfb9 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:43.140083 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1ca62b-0b43-4a45-8b04-d9dd35e41092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:43.141246 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1ca62b-0b43-4a45-8b04-d9dd35e41092 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:43.141979 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1ca62b-0b43-4a45-8b04-d9dd35e41092 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:43.142604 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1ca62b-0b43-4a45-8b04-d9dd35e41092 admin admin] PATH_INFO: `/v3/projects/8e18003e470a44cd94f7d2ee9ddcb07b` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:43.147027 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1ca62b-0b43-4a45-8b04-d9dd35e41092 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e18003e470a44cd94f7d2ee9ddcb07b)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:43.154591 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: WARNING py.warnings [None req-5e1ca62b-0b43-4a45-8b04-d9dd35e41092 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 15 19:35:43.154591 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: warnings.warn(msg) Jul 15 19:35:43.154591 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: Jul 15 19:35:43.156885 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1ca62b-0b43-4a45-8b04-d9dd35e41092 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:43.200712 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:35:43.363783 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-5e1ca62b-0b43-4a45-8b04-d9dd35e41092 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e18003e470a44cd94f7d2ee9ddcb07b', 'request_id': 'req-5e1ca62b-0b43-4a45-8b04-d9dd35e41092'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:35:43.460555 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-5e1ca62b-0b43-4a45-8b04-d9dd35e41092 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e18003e470a44cd94f7d2ee9ddcb07b', 'request_id': 'req-5e1ca62b-0b43-4a45-8b04-d9dd35e41092'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:35:43.478863 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:35:43.485857 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-5e1ca62b-0b43-4a45-8b04-d9dd35e41092 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e18003e470a44cd94f7d2ee9ddcb07b', 'request_id': 'req-5e1ca62b-0b43-4a45-8b04-d9dd35e41092'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:35:43.503147 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-5e1ca62b-0b43-4a45-8b04-d9dd35e41092 admin admin] The token cache is being invalidate because project 8e18003e470a44cd94f7d2ee9ddcb07b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:35:43.506251 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-5e1ca62b-0b43-4a45-8b04-d9dd35e41092 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:35:43.507217 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-5e1ca62b-0b43-4a45-8b04-d9dd35e41092 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:35:43.513565 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 829/1655] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jul 15 19:35:42 2022] DELETE /identity/v3/projects/8e18003e470a44cd94f7d2ee9ddcb07b => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:35:43.518634 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4966486-e24f-41de-9d10-7b0867d55638 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:43.519525 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4966486-e24f-41de-9d10-7b0867d55638 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:43.521127 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4966486-e24f-41de-9d10-7b0867d55638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:43.575008 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-c4966486-e24f-41de-9d10-7b0867d55638 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:35:43.576747 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-c4966486-e24f-41de-9d10-7b0867d55638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:43.686810 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 827/1656] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:35:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:35:43.690095 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55229644-1cf4-494b-948d-d637721dcfd0 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:43.690387 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55229644-1cf4-494b-948d-d637721dcfd0 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:43.690691 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55229644-1cf4-494b-948d-d637721dcfd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:43.716296 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-55229644-1cf4-494b-948d-d637721dcfd0 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:35:43.720250 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-55229644-1cf4-494b-948d-d637721dcfd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:43.734058 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 830/1657] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:35:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:35:43.735158 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4966486-e24f-41de-9d10-7b0867d55638 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:43.750963 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4966486-e24f-41de-9d10-7b0867d55638 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:43.765259 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b25ee7-5b05-4a10-8de3-e03a32f78241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:43.766337 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b25ee7-5b05-4a10-8de3-e03a32f78241 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:43.766872 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b25ee7-5b05-4a10-8de3-e03a32f78241 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:43.767511 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b25ee7-5b05-4a10-8de3-e03a32f78241 admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:43.768440 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b25ee7-5b05-4a10-8de3-e03a32f78241 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:35:43.768857 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b25ee7-5b05-4a10-8de3-e03a32f78241 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:43.771127 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b25ee7-5b05-4a10-8de3-e03a32f78241 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:43.798633 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55229644-1cf4-494b-948d-d637721dcfd0 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:43.803080 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 828/1658] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jul 15 19:35:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:35:43.848347 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55229644-1cf4-494b-948d-d637721dcfd0 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:43.870033 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa90c3cb-5408-4d64-8cda-c689a42687b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:43.871353 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa90c3cb-5408-4d64-8cda-c689a42687b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:43.872872 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa90c3cb-5408-4d64-8cda-c689a42687b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:43.874432 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa90c3cb-5408-4d64-8cda-c689a42687b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:43.875320 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa90c3cb-5408-4d64-8cda-c689a42687b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:43.878098 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa90c3cb-5408-4d64-8cda-c689a42687b1 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:43.903346 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b25ee7-5b05-4a10-8de3-e03a32f78241 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:43.905393 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-f5b25ee7-5b05-4a10-8de3-e03a32f78241 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:43.941023 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b25ee7-5b05-4a10-8de3-e03a32f78241 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:43.957071 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e44bd6-2802-4120-a9d5-ad0a6f28a143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:43.958146 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e44bd6-2802-4120-a9d5-ad0a6f28a143 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:43.958563 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e44bd6-2802-4120-a9d5-ad0a6f28a143 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:43.959062 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e44bd6-2802-4120-a9d5-ad0a6f28a143 admin admin] PATH_INFO: `/v3/users/72ff9ead5e494aa188af528d5dff62c9` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:43.963821 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e44bd6-2802-4120-a9d5-ad0a6f28a143 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72ff9ead5e494aa188af528d5dff62c9)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:43.965009 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: WARNING py.warnings [None req-59e44bd6-2802-4120-a9d5-ad0a6f28a143 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 15 19:35:43.965009 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: warnings.warn(msg) Jul 15 19:35:43.965009 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: Jul 15 19:35:43.981469 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e44bd6-2802-4120-a9d5-ad0a6f28a143 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:44.009952 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 831/1659] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jul 15 19:35:43 2022] POST /identity/v3/projects => generated 385 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:35:44.011960 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa90c3cb-5408-4d64-8cda-c689a42687b1 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:44.189642 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa90c3cb-5408-4d64-8cda-c689a42687b1 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:44.211034 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1254d08b-a68a-47f3-85f3-b7c8cd74c182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:44.211815 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1254d08b-a68a-47f3-85f3-b7c8cd74c182 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:44.212101 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1254d08b-a68a-47f3-85f3-b7c8cd74c182 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:44.212387 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1254d08b-a68a-47f3-85f3-b7c8cd74c182 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:44.214494 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1254d08b-a68a-47f3-85f3-b7c8cd74c182 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:44.228757 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1254d08b-a68a-47f3-85f3-b7c8cd74c182 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:44.355580 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-59e44bd6-2802-4120-a9d5-ad0a6f28a143 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72ff9ead5e494aa188af528d5dff62c9'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:35:44.403934 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 832/1660] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jul 15 19:35:44 2022] POST /identity/v3/users => generated 293 bytes in 393 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:35:44.405328 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b692614-130b-4234-8460-a858034c04ed None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:44.405621 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b692614-130b-4234-8460-a858034c04ed None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:44.434085 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b692614-130b-4234-8460-a858034c04ed None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:44.439467 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 833/1661] 10.222.0.33 () {54 vars in 912 bytes} [Fri Jul 15 19:35:44 2022] GET /identity => generated 270 bytes in 31 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:35:44.439467 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b692614-130b-4234-8460-a858034c04ed None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:44.445915 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-59e44bd6-2802-4120-a9d5-ad0a6f28a143 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72ff9ead5e494aa188af528d5dff62c9'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:35:44.480380 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b692614-130b-4234-8460-a858034c04ed None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:44.485543 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c9a0c9-000b-4661-a249-605b2b3e5676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:44.486540 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c9a0c9-000b-4661-a249-605b2b3e5676 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:44.487021 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c9a0c9-000b-4661-a249-605b2b3e5676 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:44.488772 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c9a0c9-000b-4661-a249-605b2b3e5676 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:44.490185 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c9a0c9-000b-4661-a249-605b2b3e5676 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:44.492021 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c9a0c9-000b-4661-a249-605b2b3e5676 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:44.574804 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-59e44bd6-2802-4120-a9d5-ad0a6f28a143 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72ff9ead5e494aa188af528d5dff62c9'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:35:44.575887 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-59e44bd6-2802-4120-a9d5-ad0a6f28a143 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72ff9ead5e494aa188af528d5dff62c9'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:35:44.582494 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 834/1662] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:35:44 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:35:44.587493 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5197e-66ff-4372-99b2-daa06c381ec7 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:44.587936 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5197e-66ff-4372-99b2-daa06c381ec7 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:44.588606 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5197e-66ff-4372-99b2-daa06c381ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:44.595762 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-59e44bd6-2802-4120-a9d5-ad0a6f28a143 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72ff9ead5e494aa188af528d5dff62c9'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:35:44.626258 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 829/1663] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jul 15 19:35:43 2022] DELETE /identity/v3/users/72ff9ead5e494aa188af528d5dff62c9 => generated 0 bytes in 723 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:35:44.627173 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e44bd6-2802-4120-a9d5-ad0a6f28a143 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:44.631313 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-59e44bd6-2802-4120-a9d5-ad0a6f28a143 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:44.659492 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:35:44.746059 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-1ce5197e-66ff-4372-99b2-daa06c381ec7 None None] MFA Rules not processed for user `b2a283d7d3594e37943bda4529d0a775`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:35:44.750403 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-1ce5197e-66ff-4372-99b2-daa06c381ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:44.893675 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e44bd6-2802-4120-a9d5-ad0a6f28a143 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:44.919387 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 835/1664] 10.222.0.33 () {58 vars in 1009 bytes} [Fri Jul 15 19:35:44 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:35:44.920015 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c07900-420d-4c62-98b4-6602c9b41d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:44.920015 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c07900-420d-4c62-98b4-6602c9b41d67 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:44.920015 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c07900-420d-4c62-98b4-6602c9b41d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:44.920015 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c07900-420d-4c62-98b4-6602c9b41d67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:44.920259 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c07900-420d-4c62-98b4-6602c9b41d67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:44.929365 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391f59e5-bab6-4213-8c16-a2db49bbe988 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:44.929666 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391f59e5-bab6-4213-8c16-a2db49bbe988 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:44.934496 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c07900-420d-4c62-98b4-6602c9b41d67 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:44.937423 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391f59e5-bab6-4213-8c16-a2db49bbe988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:44.965553 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 830/1665] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:35:44 2022] GET /identity/v3/roles => generated 1420 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:35:44.966708 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c07900-420d-4c62-98b4-6602c9b41d67 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:44.975936 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-71c07900-420d-4c62-98b4-6602c9b41d67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:44.989463 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-391f59e5-bab6-4213-8c16-a2db49bbe988 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:35:44.995412 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-391f59e5-bab6-4213-8c16-a2db49bbe988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:45.006188 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c07900-420d-4c62-98b4-6602c9b41d67 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:45.015737 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626b4d87-0a3e-48ca-834b-9a06c54a696e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:45.016893 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626b4d87-0a3e-48ca-834b-9a06c54a696e service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:45.017330 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626b4d87-0a3e-48ca-834b-9a06c54a696e service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:45.017917 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626b4d87-0a3e-48ca-834b-9a06c54a696e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:45.035285 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 836/1666] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:35:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:35:45.038683 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626b4d87-0a3e-48ca-834b-9a06c54a696e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:45.044270 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626b4d87-0a3e-48ca-834b-9a06c54a696e service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:45.046307 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391f59e5-bab6-4213-8c16-a2db49bbe988 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:45.082580 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 831/1667] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:35:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:35:45.083498 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626b4d87-0a3e-48ca-834b-9a06c54a696e service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:45.084031 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391f59e5-bab6-4213-8c16-a2db49bbe988 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:45.096621 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a03dd8-05c3-4012-8fc1-5d3e4da20d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:45.097537 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a03dd8-05c3-4012-8fc1-5d3e4da20d9c admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:45.097965 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a03dd8-05c3-4012-8fc1-5d3e4da20d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:45.098119 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a03dd8-05c3-4012-8fc1-5d3e4da20d9c admin admin] PATH_INFO: `/v3/projects/698fd6eba17345d59ba5d43118620c69/users/97e6d49456554a9baef702c5c3b957e2/roles/28763c6e70094116b3a4b2895895fa58` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:45.101218 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626b4d87-0a3e-48ca-834b-9a06c54a696e service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:45.118941 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41590ff4-ce8c-40d9-80eb-130f351afd9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:45.119614 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41590ff4-ce8c-40d9-80eb-130f351afd9f service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:45.119900 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41590ff4-ce8c-40d9-80eb-130f351afd9f service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:45.120202 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41590ff4-ce8c-40d9-80eb-130f351afd9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:45.148077 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a03dd8-05c3-4012-8fc1-5d3e4da20d9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=698fd6eba17345d59ba5d43118620c69, user_id=97e6d49456554a9baef702c5c3b957e2, role_id=28763c6e70094116b3a4b2895895fa58)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:45.155331 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41590ff4-ce8c-40d9-80eb-130f351afd9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:45.156797 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41590ff4-ce8c-40d9-80eb-130f351afd9f service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:45.158888 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a03dd8-05c3-4012-8fc1-5d3e4da20d9c admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:45.194822 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 832/1668] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:35:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:35:45.290323 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 837/1669] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:35:45 2022] PUT /identity/v3/projects/698fd6eba17345d59ba5d43118620c69/users/97e6d49456554a9baef702c5c3b957e2/roles/28763c6e70094116b3a4b2895895fa58 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:35:45.292128 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75d5ee6-3130-4d00-8a23-d41909467c99 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:45.292422 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75d5ee6-3130-4d00-8a23-d41909467c99 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:45.292702 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75d5ee6-3130-4d00-8a23-d41909467c99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:45.360966 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-b75d5ee6-3130-4d00-8a23-d41909467c99 None None] MFA Rules not processed for user `97e6d49456554a9baef702c5c3b957e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:35:45.377279 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b75d5ee6-3130-4d00-8a23-d41909467c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:45.594979 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 833/1670] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:35:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:35:45.597755 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a03dd8-05c3-4012-8fc1-5d3e4da20d9c admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:45.607013 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=93,keystone:DELETE=2,keystone:INSERT=5 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:35:45.782726 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed12399-f061-4e74-a36e-cf544ba36ce6 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:45.783019 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed12399-f061-4e74-a36e-cf544ba36ce6 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:45.783303 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed12399-f061-4e74-a36e-cf544ba36ce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:45.793945 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a03dd8-05c3-4012-8fc1-5d3e4da20d9c admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:45.811972 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18e6405-1c03-4869-8139-c98ae6a20217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:45.812708 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18e6405-1c03-4869-8139-c98ae6a20217 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:45.812999 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18e6405-1c03-4869-8139-c98ae6a20217 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:45.813290 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18e6405-1c03-4869-8139-c98ae6a20217 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:45.837110 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18e6405-1c03-4869-8139-c98ae6a20217 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:45.837110 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18e6405-1c03-4869-8139-c98ae6a20217 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:45.898934 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-9ed12399-f061-4e74-a36e-cf544ba36ce6 None None] MFA Rules not processed for user `eb2902b42e3449ba95b73ad5f85fcfdc`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:35:45.900202 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9ed12399-f061-4e74-a36e-cf544ba36ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:45.998232 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 838/1671] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jul 15 19:35:45 2022] POST /identity/v3/projects => generated 385 bytes in 400 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:35:46.006695 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18e6405-1c03-4869-8139-c98ae6a20217 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:46.065791 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:35:46.083976 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 834/1672] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:35:45 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:35:46.085680 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed12399-f061-4e74-a36e-cf544ba36ce6 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:46.093001 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9ed12399-f061-4e74-a36e-cf544ba36ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:46.123538 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed12399-f061-4e74-a36e-cf544ba36ce6 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:46.225081 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18e6405-1c03-4869-8139-c98ae6a20217 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:46.237650 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9c08f5-24ed-4ab7-8abc-3b32b18c63c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:46.238405 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9c08f5-24ed-4ab7-8abc-3b32b18c63c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:46.238687 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9c08f5-24ed-4ab7-8abc-3b32b18c63c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:46.238958 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9c08f5-24ed-4ab7-8abc-3b32b18c63c5 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:46.239721 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9c08f5-24ed-4ab7-8abc-3b32b18c63c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:46.241821 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9c08f5-24ed-4ab7-8abc-3b32b18c63c5 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:46.251422 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca072094-00f8-470b-a69d-b021cc9df6dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:46.252337 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca072094-00f8-470b-a69d-b021cc9df6dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:46.252626 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca072094-00f8-470b-a69d-b021cc9df6dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:46.256996 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca072094-00f8-470b-a69d-b021cc9df6dd admin admin] PATH_INFO: `/v3/users/cd105befee3d452a87501a903496e459` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:46.265312 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca072094-00f8-470b-a69d-b021cc9df6dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd105befee3d452a87501a903496e459)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:46.265581 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca072094-00f8-470b-a69d-b021cc9df6dd admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:46.426112 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 839/1673] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jul 15 19:35:46 2022] POST /identity/v3/users => generated 299 bytes in 423 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:35:46.426112 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9c08f5-24ed-4ab7-8abc-3b32b18c63c5 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:46.431919 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-5a9c08f5-24ed-4ab7-8abc-3b32b18c63c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:46.637694 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9c08f5-24ed-4ab7-8abc-3b32b18c63c5 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:46.650681 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6d296a-2143-49f3-883f-4f8bc1f31991 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7673a3b5c0324e07a9535ac70ed7a491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:46.651364 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6d296a-2143-49f3-883f-4f8bc1f31991 service nova] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:46.651657 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6d296a-2143-49f3-883f-4f8bc1f31991 service nova] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:46.651981 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6d296a-2143-49f3-883f-4f8bc1f31991 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:46.671655 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ca072094-00f8-470b-a69d-b021cc9df6dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd105befee3d452a87501a903496e459'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:35:46.697429 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ca072094-00f8-470b-a69d-b021cc9df6dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd105befee3d452a87501a903496e459'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:35:46.791893 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ca072094-00f8-470b-a69d-b021cc9df6dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd105befee3d452a87501a903496e459'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:35:46.792511 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ca072094-00f8-470b-a69d-b021cc9df6dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd105befee3d452a87501a903496e459'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:35:46.808535 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ca072094-00f8-470b-a69d-b021cc9df6dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd105befee3d452a87501a903496e459'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:35:46.819091 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6d296a-2143-49f3-883f-4f8bc1f31991 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:46.820833 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6d296a-2143-49f3-883f-4f8bc1f31991 service nova] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:46.824006 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 835/1674] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jul 15 19:35:46 2022] DELETE /identity/v3/users/cd105befee3d452a87501a903496e459 => generated 0 bytes in 739 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:35:46.825920 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986997f2-2847-4e5b-845c-c824b364b8a2 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:46.827606 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986997f2-2847-4e5b-845c-c824b364b8a2 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:46.828293 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986997f2-2847-4e5b-845c-c824b364b8a2 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:46.830316 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 836/1675] 10.222.0.33 () {54 vars in 960 bytes} [Fri Jul 15 19:35:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:35:46.831394 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986997f2-2847-4e5b-845c-c824b364b8a2 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:46.835083 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-986997f2-2847-4e5b-845c-c824b364b8a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:46.855537 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 840/1676] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:35:46 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:35:46.856939 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0278140e-3058-4540-a7cc-179e1c7f6e86 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:46.857358 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0278140e-3058-4540-a7cc-179e1c7f6e86 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:46.857656 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0278140e-3058-4540-a7cc-179e1c7f6e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:46.865617 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:35:46.904033 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-0278140e-3058-4540-a7cc-179e1c7f6e86 None None] MFA Rules not processed for user `8a654f77867341a1adf8db7f36ddc653`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:35:46.906142 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-0278140e-3058-4540-a7cc-179e1c7f6e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:46.946760 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986997f2-2847-4e5b-845c-c824b364b8a2 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:46.956305 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7d2538-d9e1-4331-8abe-f369bd27b646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:46.957449 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d2538-d9e1-4331-8abe-f369bd27b646 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:46.957973 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d2538-d9e1-4331-8abe-f369bd27b646 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:46.959009 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d2538-d9e1-4331-8abe-f369bd27b646 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:46.960537 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7d2538-d9e1-4331-8abe-f369bd27b646 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:46.964936 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7d2538-d9e1-4331-8abe-f369bd27b646 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:46.988072 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 837/1677] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:35:46 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:35:46.990756 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6e81dc-b71a-402b-aca5-fb688a09d72f None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:46.992587 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6e81dc-b71a-402b-aca5-fb688a09d72f None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:46.992587 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6e81dc-b71a-402b-aca5-fb688a09d72f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:47.014098 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 841/1678] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:35:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:35:47.016437 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6bca78-22c0-4ed9-8078-6dd48cbbfec1 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:47.017093 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6bca78-22c0-4ed9-8078-6dd48cbbfec1 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:47.017539 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6bca78-22c0-4ed9-8078-6dd48cbbfec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:47.038012 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-5f6e81dc-b71a-402b-aca5-fb688a09d72f None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:35:47.039376 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-5f6e81dc-b71a-402b-aca5-fb688a09d72f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:47.052380 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 838/1679] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:35:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:35:47.053880 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6e81dc-b71a-402b-aca5-fb688a09d72f None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:47.061414 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:35:47.074720 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6e81dc-b71a-402b-aca5-fb688a09d72f None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:47.081000 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-2e6bca78-22c0-4ed9-8078-6dd48cbbfec1 None None] MFA Rules not processed for user `52a0fce231ba4785a685ac9898d07503`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:35:47.081499 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d61ea5d-a2c4-4279-a015-ea6790ef6ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:47.082423 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-2e6bca78-22c0-4ed9-8078-6dd48cbbfec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:47.082579 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d61ea5d-a2c4-4279-a015-ea6790ef6ced admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:47.082950 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d61ea5d-a2c4-4279-a015-ea6790ef6ced admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:47.083351 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d61ea5d-a2c4-4279-a015-ea6790ef6ced admin admin] PATH_INFO: `/v3/projects/692a06b46d6447898538caea96addcb5/users/694f334e5c50485eac3c7f5b8fe61658/roles/4318f86ba9f74cd4a0bef1c311276657` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:47.103805 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d61ea5d-a2c4-4279-a015-ea6790ef6ced admin admin] RBAC: Authorizing `identity:create_grant(project_id=692a06b46d6447898538caea96addcb5, user_id=694f334e5c50485eac3c7f5b8fe61658, role_id=4318f86ba9f74cd4a0bef1c311276657)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:47.113721 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d61ea5d-a2c4-4279-a015-ea6790ef6ced admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:47.215574 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 842/1680] 10.222.0.33 () {58 vars in 1057 bytes} [Fri Jul 15 19:35:47 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:35:47.216794 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6bca78-22c0-4ed9-8078-6dd48cbbfec1 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:47.218588 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-2e6bca78-22c0-4ed9-8078-6dd48cbbfec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:47.271195 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 839/1681] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:35:47 2022] PUT /identity/v3/projects/692a06b46d6447898538caea96addcb5/users/694f334e5c50485eac3c7f5b8fe61658/roles/4318f86ba9f74cd4a0bef1c311276657 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:35:47.283553 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d61ea5d-a2c4-4279-a015-ea6790ef6ced admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:47.311935 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6bca78-22c0-4ed9-8078-6dd48cbbfec1 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:47.424976 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd3afdf-3c72-4363-a996-87d406852835 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:47.428692 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd3afdf-3c72-4363-a996-87d406852835 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:47.429550 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd3afdf-3c72-4363-a996-87d406852835 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:47.429550 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd3afdf-3c72-4363-a996-87d406852835 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:47.508664 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d61ea5d-a2c4-4279-a015-ea6790ef6ced admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:47.520503 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e09d4c-fb89-41da-aac0-b9f6f70ffd58 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52a0fce231ba4785a685ac9898d07503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:47.523254 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e09d4c-fb89-41da-aac0-b9f6f70ffd58 service placement] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:47.524932 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e09d4c-fb89-41da-aac0-b9f6f70ffd58 service placement] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:47.525208 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e09d4c-fb89-41da-aac0-b9f6f70ffd58 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:47.532248 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-c6e09d4c-fb89-41da-aac0-b9f6f70ffd58 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:47.556644 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e09d4c-fb89-41da-aac0-b9f6f70ffd58 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:47.560184 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e09d4c-fb89-41da-aac0-b9f6f70ffd58 service placement] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:47.582758 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 840/1682] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:35:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:35:47.582961 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd3afdf-3c72-4363-a996-87d406852835 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:47.584486 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd3afdf-3c72-4363-a996-87d406852835 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:47.585433 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e09d4c-fb89-41da-aac0-b9f6f70ffd58 service placement] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:47.606333 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 843/1683] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:35:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:35:47.609411 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd3afdf-3c72-4363-a996-87d406852835 service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:47.611556 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-4cd3afdf-3c72-4363-a996-87d406852835 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:47.612730 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e09d4c-fb89-41da-aac0-b9f6f70ffd58 service placement] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:47.627975 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52491462-438e-48ba-b093-d808ad31e131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:47.630427 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52491462-438e-48ba-b093-d808ad31e131 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:47.630714 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52491462-438e-48ba-b093-d808ad31e131 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:47.630992 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52491462-438e-48ba-b093-d808ad31e131 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:47.632278 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52491462-438e-48ba-b093-d808ad31e131 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:47.633984 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52491462-438e-48ba-b093-d808ad31e131 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:47.643882 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd3afdf-3c72-4363-a996-87d406852835 service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:47.660563 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 841/1684] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:35:47 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:35:47.662990 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83f98ff-5a88-4d13-84c5-c3c03d358f06 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:47.663432 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4140530-9270-43a2-aa6b-725a1e4d52f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:47.664360 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4140530-9270-43a2-aa6b-725a1e4d52f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:47.664642 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4140530-9270-43a2-aa6b-725a1e4d52f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:47.664915 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4140530-9270-43a2-aa6b-725a1e4d52f1 admin admin] PATH_INFO: `/v3/projects/16b8b4b0c5fe4769b61df425b5585ea3` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:47.665286 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83f98ff-5a88-4d13-84c5-c3c03d358f06 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:47.667855 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83f98ff-5a88-4d13-84c5-c3c03d358f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:47.667970 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4140530-9270-43a2-aa6b-725a1e4d52f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16b8b4b0c5fe4769b61df425b5585ea3)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:47.674253 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4140530-9270-43a2-aa6b-725a1e4d52f1 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:47.732680 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-d83f98ff-5a88-4d13-84c5-c3c03d358f06 None None] MFA Rules not processed for user `3d2ee510f0a14305ad13f85d2e0b3a49`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:35:47.735737 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d83f98ff-5a88-4d13-84c5-c3c03d358f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:47.840756 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:35:47.875979 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-f4140530-9270-43a2-aa6b-725a1e4d52f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16b8b4b0c5fe4769b61df425b5585ea3', 'request_id': 'req-f4140530-9270-43a2-aa6b-725a1e4d52f1'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:35:47.887491 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 842/1685] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:35:47 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:35:47.888646 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83f98ff-5a88-4d13-84c5-c3c03d358f06 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:47.920380 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83f98ff-5a88-4d13-84c5-c3c03d358f06 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:47.999344 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-f4140530-9270-43a2-aa6b-725a1e4d52f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16b8b4b0c5fe4769b61df425b5585ea3', 'request_id': 'req-f4140530-9270-43a2-aa6b-725a1e4d52f1'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:35:48.007485 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-f4140530-9270-43a2-aa6b-725a1e4d52f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16b8b4b0c5fe4769b61df425b5585ea3', 'request_id': 'req-f4140530-9270-43a2-aa6b-725a1e4d52f1'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:35:48.040369 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-f4140530-9270-43a2-aa6b-725a1e4d52f1 admin admin] The token cache is being invalidate because project 16b8b4b0c5fe4769b61df425b5585ea3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:35:48.040369 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-f4140530-9270-43a2-aa6b-725a1e4d52f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:35:48.040369 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-f4140530-9270-43a2-aa6b-725a1e4d52f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:35:48.040725 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 844/1686] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jul 15 19:35:47 2022] DELETE /identity/v3/projects/16b8b4b0c5fe4769b61df425b5585ea3 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:35:48.041743 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f8697a-caa9-4bfc-bc33-cb2487f45573 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:48.042061 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f8697a-caa9-4bfc-bc33-cb2487f45573 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:48.042344 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f8697a-caa9-4bfc-bc33-cb2487f45573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:48.090609 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1b42e6-7187-45b9-88bb-15708791c685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:48.091883 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b42e6-7187-45b9-88bb-15708791c685 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:48.092369 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b42e6-7187-45b9-88bb-15708791c685 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:48.092846 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b42e6-7187-45b9-88bb-15708791c685 admin admin] PATH_INFO: `/v3/projects/692a06b46d6447898538caea96addcb5/users/694f334e5c50485eac3c7f5b8fe61658/roles/4318f86ba9f74cd4a0bef1c311276657` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:48.097471 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1b42e6-7187-45b9-88bb-15708791c685 admin admin] RBAC: Authorizing `identity:create_grant(project_id=692a06b46d6447898538caea96addcb5, user_id=694f334e5c50485eac3c7f5b8fe61658, role_id=4318f86ba9f74cd4a0bef1c311276657)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:48.102139 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-89f8697a-caa9-4bfc-bc33-cb2487f45573 None None] MFA Rules not processed for user `3d2ee510f0a14305ad13f85d2e0b3a49`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:35:48.106590 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1b42e6-7187-45b9-88bb-15708791c685 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:48.115135 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-89f8697a-caa9-4bfc-bc33-cb2487f45573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:48.138778 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 843/1687] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:35:47 2022] PUT /identity/v3/projects/692a06b46d6447898538caea96addcb5/users/694f334e5c50485eac3c7f5b8fe61658/roles/4318f86ba9f74cd4a0bef1c311276657 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:35:48.142386 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96adaedf-437d-43a8-8801-d7b8a9e366c2 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:48.144141 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96adaedf-437d-43a8-8801-d7b8a9e366c2 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:48.144634 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96adaedf-437d-43a8-8801-d7b8a9e366c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:48.185720 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-96adaedf-437d-43a8-8801-d7b8a9e366c2 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:35:48.190191 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-96adaedf-437d-43a8-8801-d7b8a9e366c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:48.307754 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 845/1688] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:35:48 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:35:48.309111 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e73dda-ced1-4f4d-8589-4bbbb312a7e4 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:48.309404 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e73dda-ced1-4f4d-8589-4bbbb312a7e4 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:48.309682 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e73dda-ced1-4f4d-8589-4bbbb312a7e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:48.332961 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 844/1689] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:35:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:35:48.336812 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96adaedf-437d-43a8-8801-d7b8a9e366c2 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:48.359202 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-96adaedf-437d-43a8-8801-d7b8a9e366c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:48.403011 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-11e73dda-ced1-4f4d-8589-4bbbb312a7e4 None None] MFA Rules not processed for user `694f334e5c50485eac3c7f5b8fe61658`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:35:48.404465 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-11e73dda-ced1-4f4d-8589-4bbbb312a7e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:48.607246 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:35:48.633842 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 846/1690] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:35:48 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:35:48.640373 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0279a66-69c1-4092-913d-1ff620c83915 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:48.641757 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0279a66-69c1-4092-913d-1ff620c83915 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:48.642211 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0279a66-69c1-4092-913d-1ff620c83915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:48.671562 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96adaedf-437d-43a8-8801-d7b8a9e366c2 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:48.690679 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d76e84-4bdb-4b71-95f3-a86cba58595c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7673a3b5c0324e07a9535ac70ed7a491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:48.691359 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d76e84-4bdb-4b71-95f3-a86cba58595c service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:48.691645 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d76e84-4bdb-4b71-95f3-a86cba58595c service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:48.691920 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d76e84-4bdb-4b71-95f3-a86cba58595c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:48.698632 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-76d76e84-4bdb-4b71-95f3-a86cba58595c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:48.723847 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-d0279a66-69c1-4092-913d-1ff620c83915 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:35:48.726338 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-d0279a66-69c1-4092-913d-1ff620c83915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:48.754857 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 847/1691] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:35:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:35:48.759360 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0279a66-69c1-4092-913d-1ff620c83915 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:48.762741 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-d0279a66-69c1-4092-913d-1ff620c83915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:48.869697 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d76e84-4bdb-4b71-95f3-a86cba58595c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:48.872318 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d76e84-4bdb-4b71-95f3-a86cba58595c service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:48.885740 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 845/1692] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:35:48 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 552 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:35:48.887713 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d76e84-4bdb-4b71-95f3-a86cba58595c service nova] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:48.902376 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d76e84-4bdb-4b71-95f3-a86cba58595c service nova] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:48.916358 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d93d988-32f3-4be3-a31f-d30ea5e0686c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:48.932534 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d93d988-32f3-4be3-a31f-d30ea5e0686c admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:48.933082 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d93d988-32f3-4be3-a31f-d30ea5e0686c admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:48.933494 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d93d988-32f3-4be3-a31f-d30ea5e0686c admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:48.936277 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d93d988-32f3-4be3-a31f-d30ea5e0686c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:35:48.936776 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d93d988-32f3-4be3-a31f-d30ea5e0686c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:48.940947 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d93d988-32f3-4be3-a31f-d30ea5e0686c admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:48.942388 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0279a66-69c1-4092-913d-1ff620c83915 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:48.957232 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e97147c-5153-439f-b7e6-d76a63b1fb10 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86c2df3d378490ca7ea497a73c19d40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:48.958529 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e97147c-5153-439f-b7e6-d76a63b1fb10 service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:48.959103 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e97147c-5153-439f-b7e6-d76a63b1fb10 service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:48.959630 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e97147c-5153-439f-b7e6-d76a63b1fb10 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:48.988045 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 846/1693] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jul 15 19:35:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:35:48.990320 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d93d988-32f3-4be3-a31f-d30ea5e0686c admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:48.997045 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-8d93d988-32f3-4be3-a31f-d30ea5e0686c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:48.997975 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e97147c-5153-439f-b7e6-d76a63b1fb10 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:49.006378 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e97147c-5153-439f-b7e6-d76a63b1fb10 service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:49.022310 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:35:49.027088 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d93d988-32f3-4be3-a31f-d30ea5e0686c admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:49.032787 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ce45d9-860f-44cb-8075-ad3de6c8805d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:49.037131 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce45d9-860f-44cb-8075-ad3de6c8805d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:49.037507 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce45d9-860f-44cb-8075-ad3de6c8805d admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:49.037822 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce45d9-860f-44cb-8075-ad3de6c8805d admin admin] PATH_INFO: `/v3/projects/a84088d3693f448dadc1c58699966ed6` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:49.039371 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ce45d9-860f-44cb-8075-ad3de6c8805d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a84088d3693f448dadc1c58699966ed6)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:49.041084 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 848/1694] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:35:48 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:35:49.041652 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ce45d9-860f-44cb-8075-ad3de6c8805d admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:49.042213 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8526b9ed-c591-477c-81f8-121a42158303 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:49.042502 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8526b9ed-c591-477c-81f8-121a42158303 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:49.042769 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8526b9ed-c591-477c-81f8-121a42158303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:49.082864 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-8526b9ed-c591-477c-81f8-121a42158303 None None] MFA Rules not processed for user `6193fdc75def489e88eda70e78ec6f46`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:35:49.084195 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-8526b9ed-c591-477c-81f8-121a42158303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:49.173724 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-08ce45d9-860f-44cb-8075-ad3de6c8805d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a84088d3693f448dadc1c58699966ed6', 'request_id': 'req-08ce45d9-860f-44cb-8075-ad3de6c8805d'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:35:49.268550 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-08ce45d9-860f-44cb-8075-ad3de6c8805d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a84088d3693f448dadc1c58699966ed6', 'request_id': 'req-08ce45d9-860f-44cb-8075-ad3de6c8805d'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:35:49.281153 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-08ce45d9-860f-44cb-8075-ad3de6c8805d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a84088d3693f448dadc1c58699966ed6', 'request_id': 'req-08ce45d9-860f-44cb-8075-ad3de6c8805d'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:35:49.287154 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 849/1695] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:35:49 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:35:49.288200 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8526b9ed-c591-477c-81f8-121a42158303 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:49.314647 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-08ce45d9-860f-44cb-8075-ad3de6c8805d admin admin] The token cache is being invalidate because project a84088d3693f448dadc1c58699966ed6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:35:49.317734 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-08ce45d9-860f-44cb-8075-ad3de6c8805d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:35:49.320759 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-08ce45d9-860f-44cb-8075-ad3de6c8805d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:35:49.328884 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 847/1696] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jul 15 19:35:48 2022] DELETE /identity/v3/projects/a84088d3693f448dadc1c58699966ed6 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:35:49.331072 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ce45d9-860f-44cb-8075-ad3de6c8805d admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:49.334782 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-08ce45d9-860f-44cb-8075-ad3de6c8805d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:49.342441 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8526b9ed-c591-477c-81f8-121a42158303 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:49.554678 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ce45d9-860f-44cb-8075-ad3de6c8805d admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:49.568157 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d296dc-20ee-49d1-bd2a-973e76eafd3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:49.568365 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e01f6c5-1ca8-43a7-b206-4373d7f343df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:49.569197 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01f6c5-1ca8-43a7-b206-4373d7f343df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:49.569538 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01f6c5-1ca8-43a7-b206-4373d7f343df admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:49.569738 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d296dc-20ee-49d1-bd2a-973e76eafd3d admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:49.569867 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01f6c5-1ca8-43a7-b206-4373d7f343df admin admin] PATH_INFO: `/v3/users/97e6d49456554a9baef702c5c3b957e2` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:49.570267 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d296dc-20ee-49d1-bd2a-973e76eafd3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:49.571940 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e01f6c5-1ca8-43a7-b206-4373d7f343df admin admin] RBAC: Authorizing `identity:delete_user(user_id=97e6d49456554a9baef702c5c3b957e2)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:49.573604 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d296dc-20ee-49d1-bd2a-973e76eafd3d admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:49.574693 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d296dc-20ee-49d1-bd2a-973e76eafd3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:49.578141 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e01f6c5-1ca8-43a7-b206-4373d7f343df admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:49.583881 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d296dc-20ee-49d1-bd2a-973e76eafd3d admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:49.708923 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 850/1697] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jul 15 19:35:49 2022] POST /identity/v3/projects => generated 355 bytes in 420 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:35:49.709345 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd1b2c2-effd-460e-b657-ff43d4967987 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:49.714155 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd1b2c2-effd-460e-b657-ff43d4967987 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:49.714155 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd1b2c2-effd-460e-b657-ff43d4967987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:49.758471 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-bbd1b2c2-effd-460e-b657-ff43d4967987 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:35:49.760215 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-bbd1b2c2-effd-460e-b657-ff43d4967987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:49.867675 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 851/1698] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:35:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:35:49.868831 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd1b2c2-effd-460e-b657-ff43d4967987 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:49.875814 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-bbd1b2c2-effd-460e-b657-ff43d4967987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:49.894071 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd1b2c2-effd-460e-b657-ff43d4967987 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:49.901360 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57df97bd-c681-4eb4-a5fc-8048254fdfa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:49.902419 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57df97bd-c681-4eb4-a5fc-8048254fdfa1 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:49.902857 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57df97bd-c681-4eb4-a5fc-8048254fdfa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:49.903284 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57df97bd-c681-4eb4-a5fc-8048254fdfa1 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:49.904761 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57df97bd-c681-4eb4-a5fc-8048254fdfa1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:49.907622 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57df97bd-c681-4eb4-a5fc-8048254fdfa1 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:49.972175 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4e01f6c5-1ca8-43a7-b206-4373d7f343df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97e6d49456554a9baef702c5c3b957e2'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:35:49.988238 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4e01f6c5-1ca8-43a7-b206-4373d7f343df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97e6d49456554a9baef702c5c3b957e2'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:35:50.080283 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4e01f6c5-1ca8-43a7-b206-4373d7f343df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97e6d49456554a9baef702c5c3b957e2'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:35:50.081194 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4e01f6c5-1ca8-43a7-b206-4373d7f343df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97e6d49456554a9baef702c5c3b957e2'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:35:50.092802 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4e01f6c5-1ca8-43a7-b206-4373d7f343df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97e6d49456554a9baef702c5c3b957e2'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:35:50.096134 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 852/1699] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jul 15 19:35:49 2022] POST /identity/v3/users => generated 278 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:35:50.106079 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dd9489-7385-41ff-8358-9b7d01eb99a6 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:50.106573 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 848/1700] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jul 15 19:35:49 2022] DELETE /identity/v3/users/97e6d49456554a9baef702c5c3b957e2 => generated 0 bytes in 774 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:35:50.108314 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dd9489-7385-41ff-8358-9b7d01eb99a6 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:50.108564 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa61083-e550-401a-bee5-00da7b2be6fa None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:50.108673 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dd9489-7385-41ff-8358-9b7d01eb99a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:50.109111 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa61083-e550-401a-bee5-00da7b2be6fa None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:50.109726 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa61083-e550-401a-bee5-00da7b2be6fa None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:50.111956 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 849/1701] 10.222.0.33 () {54 vars in 912 bytes} [Fri Jul 15 19:35:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:35:50.112940 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa61083-e550-401a-bee5-00da7b2be6fa None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:50.115118 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-cfa61083-e550-401a-bee5-00da7b2be6fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:50.157396 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-41dd9489-7385-41ff-8358-9b7d01eb99a6 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:35:50.158668 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-41dd9489-7385-41ff-8358-9b7d01eb99a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:50.266115 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa61083-e550-401a-bee5-00da7b2be6fa None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:50.271760 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc013e5-3b99-4010-a6c6-e06b27ed39b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:50.272471 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc013e5-3b99-4010-a6c6-e06b27ed39b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:50.275420 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc013e5-3b99-4010-a6c6-e06b27ed39b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:50.275420 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc013e5-3b99-4010-a6c6-e06b27ed39b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:50.275420 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc013e5-3b99-4010-a6c6-e06b27ed39b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:50.275812 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc013e5-3b99-4010-a6c6-e06b27ed39b7 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:50.286326 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 853/1702] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:35:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:35:50.287623 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5b740b-b65a-4912-9252-c04ab35d2708 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:50.287902 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5b740b-b65a-4912-9252-c04ab35d2708 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:50.288182 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5b740b-b65a-4912-9252-c04ab35d2708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:50.292861 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 850/1703] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:35:50 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:35:50.292861 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf38aa2-8561-42ac-9494-b837b927037a None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:50.292861 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf38aa2-8561-42ac-9494-b837b927037a None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:50.292861 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf38aa2-8561-42ac-9494-b837b927037a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:50.315297 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-caf38aa2-8561-42ac-9494-b837b927037a None None] MFA Rules not processed for user `b2a283d7d3594e37943bda4529d0a775`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:35:50.316921 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-caf38aa2-8561-42ac-9494-b837b927037a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:50.329620 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-ec5b740b-b65a-4912-9252-c04ab35d2708 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:35:50.330890 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-ec5b740b-b65a-4912-9252-c04ab35d2708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:50.371610 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 854/1704] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:35:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:35:50.373992 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5b740b-b65a-4912-9252-c04ab35d2708 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:50.392249 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5b740b-b65a-4912-9252-c04ab35d2708 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:50.414659 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bd86ff-539b-4aba-92be-7065ee5faf4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:50.415481 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bd86ff-539b-4aba-92be-7065ee5faf4f admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:50.416695 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bd86ff-539b-4aba-92be-7065ee5faf4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:50.417890 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bd86ff-539b-4aba-92be-7065ee5faf4f admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:50.418727 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bd86ff-539b-4aba-92be-7065ee5faf4f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:35:50.420440 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bd86ff-539b-4aba-92be-7065ee5faf4f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:50.424579 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bd86ff-539b-4aba-92be-7065ee5faf4f admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:50.441713 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 851/1705] 10.222.0.33 () {58 vars in 1009 bytes} [Fri Jul 15 19:35:50 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:35:50.444190 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf38aa2-8561-42ac-9494-b837b927037a None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:50.445711 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 855/1706] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jul 15 19:35:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:35:50.452658 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bd86ff-539b-4aba-92be-7065ee5faf4f admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:50.454543 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-07bd86ff-539b-4aba-92be-7065ee5faf4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:50.464212 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf38aa2-8561-42ac-9494-b837b927037a None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:50.470838 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bd86ff-539b-4aba-92be-7065ee5faf4f admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:50.476894 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e923e178-7623-4580-ac89-7fa69813b440 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:50.477653 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923e178-7623-4580-ac89-7fa69813b440 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:50.478190 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923e178-7623-4580-ac89-7fa69813b440 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:50.478547 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923e178-7623-4580-ac89-7fa69813b440 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:50.478941 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb0a465-d51e-4fee-a5f0-eba03e990dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:50.482775 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb0a465-d51e-4fee-a5f0-eba03e990dc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:50.483318 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb0a465-d51e-4fee-a5f0-eba03e990dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:50.483821 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb0a465-d51e-4fee-a5f0-eba03e990dc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:50.485244 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb0a465-d51e-4fee-a5f0-eba03e990dc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:50.488509 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb0a465-d51e-4fee-a5f0-eba03e990dc6 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:50.491285 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e923e178-7623-4580-ac89-7fa69813b440 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:50.493038 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e923e178-7623-4580-ac89-7fa69813b440 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:50.504285 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 852/1707] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:35:50 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:35:50.508925 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 856/1708] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:35:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:35:50.510455 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e923e178-7623-4580-ac89-7fa69813b440 service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:50.510622 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb0a465-d51e-4fee-a5f0-eba03e990dc6 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:50.525583 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb0a465-d51e-4fee-a5f0-eba03e990dc6 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:50.532945 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd96781-1139-45cd-8500-f86902c124f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:50.534032 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd96781-1139-45cd-8500-f86902c124f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:50.534619 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd96781-1139-45cd-8500-f86902c124f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:50.535043 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd96781-1139-45cd-8500-f86902c124f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:50.536855 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd96781-1139-45cd-8500-f86902c124f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:50.540407 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e923e178-7623-4580-ac89-7fa69813b440 service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:50.541165 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd96781-1139-45cd-8500-f86902c124f3 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:50.556708 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:35:50.557001 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c4aacc-16b3-4056-8dc7-417d2f38f7cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:50.565910 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c4aacc-16b3-4056-8dc7-417d2f38f7cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:50.566672 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c4aacc-16b3-4056-8dc7-417d2f38f7cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:50.567106 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c4aacc-16b3-4056-8dc7-417d2f38f7cc admin admin] PATH_INFO: `/v3/projects/0d6b04e5b04f4b00be9b74a05d74ee04/users/dca75d34e96d49d5bff2b15c4fb26fae/roles/28763c6e70094116b3a4b2895895fa58` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:50.608283 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c4aacc-16b3-4056-8dc7-417d2f38f7cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d6b04e5b04f4b00be9b74a05d74ee04, user_id=dca75d34e96d49d5bff2b15c4fb26fae, role_id=28763c6e70094116b3a4b2895895fa58)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:50.611486 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c4aacc-16b3-4056-8dc7-417d2f38f7cc admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:50.803337 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 857/1709] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:35:50 2022] PUT /identity/v3/projects/0d6b04e5b04f4b00be9b74a05d74ee04/users/dca75d34e96d49d5bff2b15c4fb26fae/roles/28763c6e70094116b3a4b2895895fa58 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:35:50.806152 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 853/1710] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jul 15 19:35:50 2022] POST /identity/v3/projects => generated 403 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:35:50.807495 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c4aacc-16b3-4056-8dc7-417d2f38f7cc admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:50.812531 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd1587-7362-4196-9f99-6e1221e5b8fd None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:50.814965 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd1587-7362-4196-9f99-6e1221e5b8fd None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:50.815841 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd1587-7362-4196-9f99-6e1221e5b8fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:50.902932 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-c2dd1587-7362-4196-9f99-6e1221e5b8fd None None] MFA Rules not processed for user `dca75d34e96d49d5bff2b15c4fb26fae`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:35:50.914769 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-c2dd1587-7362-4196-9f99-6e1221e5b8fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:50.922398 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c4aacc-16b3-4056-8dc7-417d2f38f7cc admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:51.015310 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e535fc5-3cc4-45f0-88c4-4655af69b231 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:51.016327 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e535fc5-3cc4-45f0-88c4-4655af69b231 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:51.017174 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e535fc5-3cc4-45f0-88c4-4655af69b231 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:51.017620 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e535fc5-3cc4-45f0-88c4-4655af69b231 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:51.052909 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e535fc5-3cc4-45f0-88c4-4655af69b231 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:51.054859 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e535fc5-3cc4-45f0-88c4-4655af69b231 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:51.103194 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 858/1711] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:35:50 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:35:51.105911 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e535fc5-3cc4-45f0-88c4-4655af69b231 service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:51.163184 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 854/1712] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:35:50 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:35:51.167890 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dd1587-7362-4196-9f99-6e1221e5b8fd None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:51.313346 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:35:51.441335 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e535fc5-3cc4-45f0-88c4-4655af69b231 service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:51.466541 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cca6185-c2f1-4692-80bb-1ce37db7515d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:51.470121 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cca6185-c2f1-4692-80bb-1ce37db7515d admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:51.470764 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cca6185-c2f1-4692-80bb-1ce37db7515d admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:51.470764 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cca6185-c2f1-4692-80bb-1ce37db7515d admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:51.475792 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cca6185-c2f1-4692-80bb-1ce37db7515d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:51.480675 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dd1587-7362-4196-9f99-6e1221e5b8fd None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:51.486690 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cca6185-c2f1-4692-80bb-1ce37db7515d admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:51.517822 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636f8978-d85c-4ebb-a85e-b7c42c996c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:51.518571 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636f8978-d85c-4ebb-a85e-b7c42c996c3a admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:51.519027 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636f8978-d85c-4ebb-a85e-b7c42c996c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:51.519306 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636f8978-d85c-4ebb-a85e-b7c42c996c3a admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:51.523952 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636f8978-d85c-4ebb-a85e-b7c42c996c3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:51.525965 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636f8978-d85c-4ebb-a85e-b7c42c996c3a admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:51.620083 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 859/1713] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jul 15 19:35:51 2022] POST /identity/v3/users => generated 302 bytes in 516 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:35:51.623822 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cca6185-c2f1-4692-80bb-1ce37db7515d admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:51.654825 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cca6185-c2f1-4692-80bb-1ce37db7515d admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:51.659131 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 855/1714] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jul 15 19:35:51 2022] POST /identity/v3/projects => generated 357 bytes in 491 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:35:51.665782 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636f8978-d85c-4ebb-a85e-b7c42c996c3a admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:51.675329 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d3496c-a935-474e-8093-2f2de12548a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:51.676033 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d3496c-a935-474e-8093-2f2de12548a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:51.676313 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d3496c-a935-474e-8093-2f2de12548a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:51.700217 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d3496c-a935-474e-8093-2f2de12548a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:51.701103 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d3496c-a935-474e-8093-2f2de12548a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:51.702809 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d3496c-a935-474e-8093-2f2de12548a6 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:51.767635 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 860/1715] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:35:51 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 19:35:51.778168 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d3496c-a935-474e-8093-2f2de12548a6 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:51.801349 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d3496c-a935-474e-8093-2f2de12548a6 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:51.892193 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636f8978-d85c-4ebb-a85e-b7c42c996c3a admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:51.912968 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f5d3eb-b818-4072-8c1d-55d0404683e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:51.914017 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f5d3eb-b818-4072-8c1d-55d0404683e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:51.915297 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f5d3eb-b818-4072-8c1d-55d0404683e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:51.915728 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f5d3eb-b818-4072-8c1d-55d0404683e7 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:51.917974 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f5d3eb-b818-4072-8c1d-55d0404683e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:51.923110 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f5d3eb-b818-4072-8c1d-55d0404683e7 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:51.970136 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505b03e3-b517-4e9b-b42b-732168da583e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:51.971345 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505b03e3-b517-4e9b-b42b-732168da583e admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:51.971762 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505b03e3-b517-4e9b-b42b-732168da583e admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:51.972144 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505b03e3-b517-4e9b-b42b-732168da583e admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:51.973350 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505b03e3-b517-4e9b-b42b-732168da583e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:51.975753 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505b03e3-b517-4e9b-b42b-732168da583e admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:51.988931 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 861/1716] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:35:51 2022] GET /identity/v3/roles => generated 1420 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:35:52.001614 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505b03e3-b517-4e9b-b42b-732168da583e admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:52.016714 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505b03e3-b517-4e9b-b42b-732168da583e admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:52.040472 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca22685-bfc7-497c-88b9-bd36e39c0543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:52.045604 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca22685-bfc7-497c-88b9-bd36e39c0543 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:52.045943 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca22685-bfc7-497c-88b9-bd36e39c0543 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:52.046230 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca22685-bfc7-497c-88b9-bd36e39c0543 admin admin] PATH_INFO: `/v3/projects/43ec1f1f67f84fdf96cb309e823df761/users/77dd6b8d55ae4eddad0b5d8cf0f389c1/roles/28763c6e70094116b3a4b2895895fa58` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:52.050952 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 856/1717] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jul 15 19:35:51 2022] POST /identity/v3/users => generated 285 bytes in 387 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:35:52.062987 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f5d3eb-b818-4072-8c1d-55d0404683e7 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:52.081800 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca22685-bfc7-497c-88b9-bd36e39c0543 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43ec1f1f67f84fdf96cb309e823df761, user_id=77dd6b8d55ae4eddad0b5d8cf0f389c1, role_id=28763c6e70094116b3a4b2895895fa58)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:52.084276 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca22685-bfc7-497c-88b9-bd36e39c0543 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:52.092238 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f5d3eb-b818-4072-8c1d-55d0404683e7 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:52.141169 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25b3bce-063a-41ec-a80a-43a9d7835596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:52.143497 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b3bce-063a-41ec-a80a-43a9d7835596 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:52.144012 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b3bce-063a-41ec-a80a-43a9d7835596 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:52.144723 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b3bce-063a-41ec-a80a-43a9d7835596 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:52.145955 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25b3bce-063a-41ec-a80a-43a9d7835596 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:52.148135 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25b3bce-063a-41ec-a80a-43a9d7835596 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:52.178606 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 857/1718] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:35:52 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:35:52.179867 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25b3bce-063a-41ec-a80a-43a9d7835596 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:52.185449 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d25b3bce-063a-41ec-a80a-43a9d7835596 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:52.205411 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 862/1719] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:35:52 2022] PUT /identity/v3/projects/43ec1f1f67f84fdf96cb309e823df761/users/77dd6b8d55ae4eddad0b5d8cf0f389c1/roles/28763c6e70094116b3a4b2895895fa58 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:35:52.207530 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca22685-bfc7-497c-88b9-bd36e39c0543 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:52.213604 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25b3bce-063a-41ec-a80a-43a9d7835596 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:52.266650 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:35:52.362615 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca22685-bfc7-497c-88b9-bd36e39c0543 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:52.371650 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd549888-99e4-42c7-9311-5b473567d378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:52.372648 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd549888-99e4-42c7-9311-5b473567d378 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:52.372996 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd549888-99e4-42c7-9311-5b473567d378 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:52.373284 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd549888-99e4-42c7-9311-5b473567d378 admin admin] PATH_INFO: `/v3/projects/a9b79cb0f5004bbd91601bd1e4ddf892/users/611152a8389a436884553fbaba6b7447/roles/4318f86ba9f74cd4a0bef1c311276657` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:52.424258 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd549888-99e4-42c7-9311-5b473567d378 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9b79cb0f5004bbd91601bd1e4ddf892, user_id=611152a8389a436884553fbaba6b7447, role_id=4318f86ba9f74cd4a0bef1c311276657)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:52.436267 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd549888-99e4-42c7-9311-5b473567d378 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:52.439345 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb1dcee-7027-426d-bc36-08f623e2340e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:52.440353 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1dcee-7027-426d-bc36-08f623e2340e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:52.440646 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1dcee-7027-426d-bc36-08f623e2340e admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:52.440930 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1dcee-7027-426d-bc36-08f623e2340e admin admin] PATH_INFO: `/v3/projects/698fd6eba17345d59ba5d43118620c69` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:52.443554 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb1dcee-7027-426d-bc36-08f623e2340e admin admin] RBAC: Authorizing `identity:delete_project(project_id=698fd6eba17345d59ba5d43118620c69)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:52.445717 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb1dcee-7027-426d-bc36-08f623e2340e admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:52.526483 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 863/1720] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:35:52 2022] PUT /identity/v3/projects/a9b79cb0f5004bbd91601bd1e4ddf892/users/611152a8389a436884553fbaba6b7447/roles/4318f86ba9f74cd4a0bef1c311276657 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:35:52.533890 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4374d3df-73a7-4eee-bb64-e9cc98be80fd None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:52.546035 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4374d3df-73a7-4eee-bb64-e9cc98be80fd None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:52.546397 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4374d3df-73a7-4eee-bb64-e9cc98be80fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:52.659988 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-4374d3df-73a7-4eee-bb64-e9cc98be80fd None None] MFA Rules not processed for user `77dd6b8d55ae4eddad0b5d8cf0f389c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:35:52.663397 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-1cb1dcee-7027-426d-bc36-08f623e2340e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '698fd6eba17345d59ba5d43118620c69', 'request_id': 'req-1cb1dcee-7027-426d-bc36-08f623e2340e'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:35:52.665001 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-4374d3df-73a7-4eee-bb64-e9cc98be80fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:52.674680 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:35:52.765208 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-1cb1dcee-7027-426d-bc36-08f623e2340e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '698fd6eba17345d59ba5d43118620c69', 'request_id': 'req-1cb1dcee-7027-426d-bc36-08f623e2340e'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:35:52.776072 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-1cb1dcee-7027-426d-bc36-08f623e2340e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '698fd6eba17345d59ba5d43118620c69', 'request_id': 'req-1cb1dcee-7027-426d-bc36-08f623e2340e'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:35:52.821370 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-1cb1dcee-7027-426d-bc36-08f623e2340e admin admin] The token cache is being invalidate because project 698fd6eba17345d59ba5d43118620c69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:35:52.821693 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-1cb1dcee-7027-426d-bc36-08f623e2340e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:35:52.827780 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-1cb1dcee-7027-426d-bc36-08f623e2340e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:35:52.829707 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 858/1721] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jul 15 19:35:52 2022] DELETE /identity/v3/projects/698fd6eba17345d59ba5d43118620c69 => generated 0 bytes in 651 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:35:52.836101 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb1dcee-7027-426d-bc36-08f623e2340e admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:52.840586 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-1cb1dcee-7027-426d-bc36-08f623e2340e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:52.966955 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 864/1722] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:35:52 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 440 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:35:52.972581 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4374d3df-73a7-4eee-bb64-e9cc98be80fd None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:52.976875 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-4374d3df-73a7-4eee-bb64-e9cc98be80fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:53.080709 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb1dcee-7027-426d-bc36-08f623e2340e admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:53.099802 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f40307b-b914-4622-8003-5ba4b92ccc3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:53.100796 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f40307b-b914-4622-8003-5ba4b92ccc3d admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:53.102235 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f40307b-b914-4622-8003-5ba4b92ccc3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:53.102235 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f40307b-b914-4622-8003-5ba4b92ccc3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:53.105086 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f40307b-b914-4622-8003-5ba4b92ccc3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:53.111647 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f40307b-b914-4622-8003-5ba4b92ccc3d admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:53.141409 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 859/1723] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:35:52 2022] GET /identity/v3/roles => generated 1420 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:35:53.143597 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f40307b-b914-4622-8003-5ba4b92ccc3d admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:53.155976 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-7f40307b-b914-4622-8003-5ba4b92ccc3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:53.184489 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f40307b-b914-4622-8003-5ba4b92ccc3d admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:53.208120 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505cf9c3-82b4-43e9-8882-c9bc5b0b9ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:53.208915 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505cf9c3-82b4-43e9-8882-c9bc5b0b9ab1 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:53.209296 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505cf9c3-82b4-43e9-8882-c9bc5b0b9ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:53.209684 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505cf9c3-82b4-43e9-8882-c9bc5b0b9ab1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:53.210512 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505cf9c3-82b4-43e9-8882-c9bc5b0b9ab1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:53.212559 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505cf9c3-82b4-43e9-8882-c9bc5b0b9ab1 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:53.310788 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4374d3df-73a7-4eee-bb64-e9cc98be80fd None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:53.338263 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21918646-65ce-48d6-a021-233e0a0a3ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:53.341131 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21918646-65ce-48d6-a021-233e0a0a3ac5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:53.341131 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21918646-65ce-48d6-a021-233e0a0a3ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:53.343281 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21918646-65ce-48d6-a021-233e0a0a3ac5 admin admin] PATH_INFO: `/v3/users/694f334e5c50485eac3c7f5b8fe61658` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:53.345936 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21918646-65ce-48d6-a021-233e0a0a3ac5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=694f334e5c50485eac3c7f5b8fe61658)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:53.350557 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21918646-65ce-48d6-a021-233e0a0a3ac5 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:53.429706 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 860/1724] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jul 15 19:35:53 2022] POST /identity/v3/projects => generated 401 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:35:53.430788 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505cf9c3-82b4-43e9-8882-c9bc5b0b9ab1 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:53.635846 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505cf9c3-82b4-43e9-8882-c9bc5b0b9ab1 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:53.648777 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dd904f-d1ae-4ef4-9917-c99d0f6c7434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:53.651514 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd904f-d1ae-4ef4-9917-c99d0f6c7434 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:53.651514 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd904f-d1ae-4ef4-9917-c99d0f6c7434 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:53.651514 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd904f-d1ae-4ef4-9917-c99d0f6c7434 admin admin] PATH_INFO: `/v3/projects/a9b79cb0f5004bbd91601bd1e4ddf892/users/611152a8389a436884553fbaba6b7447/roles/4318f86ba9f74cd4a0bef1c311276657` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:53.657657 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dd904f-d1ae-4ef4-9917-c99d0f6c7434 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9b79cb0f5004bbd91601bd1e4ddf892, user_id=611152a8389a436884553fbaba6b7447, role_id=4318f86ba9f74cd4a0bef1c311276657)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:53.663441 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dd904f-d1ae-4ef4-9917-c99d0f6c7434 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:53.700113 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 861/1725] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:35:53 2022] PUT /identity/v3/projects/a9b79cb0f5004bbd91601bd1e4ddf892/users/611152a8389a436884553fbaba6b7447/roles/4318f86ba9f74cd4a0bef1c311276657 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:35:53.701406 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dd904f-d1ae-4ef4-9917-c99d0f6c7434 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:53.767989 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-21918646-65ce-48d6-a021-233e0a0a3ac5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '694f334e5c50485eac3c7f5b8fe61658'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:35:53.800324 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-21918646-65ce-48d6-a021-233e0a0a3ac5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '694f334e5c50485eac3c7f5b8fe61658'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:35:53.856712 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dd904f-d1ae-4ef4-9917-c99d0f6c7434 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:53.862815 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bc62f6-42aa-43f1-8d0c-11baf56ffa41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:53.870181 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bc62f6-42aa-43f1-8d0c-11baf56ffa41 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:53.870181 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bc62f6-42aa-43f1-8d0c-11baf56ffa41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:53.870181 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bc62f6-42aa-43f1-8d0c-11baf56ffa41 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:53.870181 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bc62f6-42aa-43f1-8d0c-11baf56ffa41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:53.870181 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bc62f6-42aa-43f1-8d0c-11baf56ffa41 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:53.946372 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-21918646-65ce-48d6-a021-233e0a0a3ac5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '694f334e5c50485eac3c7f5b8fe61658'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:35:53.947269 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-21918646-65ce-48d6-a021-233e0a0a3ac5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '694f334e5c50485eac3c7f5b8fe61658'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:35:53.948473 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-21918646-65ce-48d6-a021-233e0a0a3ac5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '694f334e5c50485eac3c7f5b8fe61658'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:35:53.974883 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 865/1726] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jul 15 19:35:52 2022] DELETE /identity/v3/users/694f334e5c50485eac3c7f5b8fe61658 => generated 0 bytes in 1007 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:35:53.977084 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163301cd-e73f-4a09-82ab-5d62b1ff5434 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:53.977528 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163301cd-e73f-4a09-82ab-5d62b1ff5434 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:53.977975 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163301cd-e73f-4a09-82ab-5d62b1ff5434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:53.987703 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:35:53.999714 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 862/1727] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jul 15 19:35:53 2022] POST /identity/v3/users => generated 307 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:35:53.999714 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7a6694-7127-49d6-8e6d-1ab68acb9b9b None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:53.999714 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7a6694-7127-49d6-8e6d-1ab68acb9b9b None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:53.999714 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7a6694-7127-49d6-8e6d-1ab68acb9b9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:54.045635 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-163301cd-e73f-4a09-82ab-5d62b1ff5434 None None] MFA Rules not processed for user `3d893c14620b4a129842062f14a68a78`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:35:54.047159 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-163301cd-e73f-4a09-82ab-5d62b1ff5434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:54.050680 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-0b7a6694-7127-49d6-8e6d-1ab68acb9b9b None None] MFA Rules not processed for user `611152a8389a436884553fbaba6b7447`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:35:54.052608 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0b7a6694-7127-49d6-8e6d-1ab68acb9b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:54.147325 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 866/1728] 10.222.0.33 () {58 vars in 1009 bytes} [Fri Jul 15 19:35:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:35:54.148715 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea29fe4c-d85d-4e9f-89d8-f406d3938d96 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:54.149010 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea29fe4c-d85d-4e9f-89d8-f406d3938d96 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:54.149289 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea29fe4c-d85d-4e9f-89d8-f406d3938d96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:54.176403 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-ea29fe4c-d85d-4e9f-89d8-f406d3938d96 None None] MFA Rules not processed for user `eb2902b42e3449ba95b73ad5f85fcfdc`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:35:54.176403 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-ea29fe4c-d85d-4e9f-89d8-f406d3938d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:54.184706 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 863/1729] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:35:53 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:35:54.185745 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7a6694-7127-49d6-8e6d-1ab68acb9b9b None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:54.189837 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0b7a6694-7127-49d6-8e6d-1ab68acb9b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:54.307381 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 867/1730] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:35:54 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:35:54.308283 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea29fe4c-d85d-4e9f-89d8-f406d3938d96 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:54.309604 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-ea29fe4c-d85d-4e9f-89d8-f406d3938d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:54.406248 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7a6694-7127-49d6-8e6d-1ab68acb9b9b None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:54.419921 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa0e15b-e891-496b-801c-7341bcc8c257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:54.421144 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa0e15b-e891-496b-801c-7341bcc8c257 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:54.421539 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa0e15b-e891-496b-801c-7341bcc8c257 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:54.423239 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa0e15b-e891-496b-801c-7341bcc8c257 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:54.424346 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa0e15b-e891-496b-801c-7341bcc8c257 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:54.431543 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa0e15b-e891-496b-801c-7341bcc8c257 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:54.441021 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea29fe4c-d85d-4e9f-89d8-f406d3938d96 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:54.449809 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d146c1-1d9d-4e9f-8fae-e5a287f65f13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86c2df3d378490ca7ea497a73c19d40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:54.450488 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d146c1-1d9d-4e9f-8fae-e5a287f65f13 service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:54.450766 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d146c1-1d9d-4e9f-8fae-e5a287f65f13 service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:54.451040 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d146c1-1d9d-4e9f-8fae-e5a287f65f13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:54.463644 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 864/1731] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:35:54 2022] GET /identity/v3/roles => generated 1420 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:35:54.465954 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa0e15b-e891-496b-801c-7341bcc8c257 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:54.478599 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d146c1-1d9d-4e9f-8fae-e5a287f65f13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:54.482159 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d146c1-1d9d-4e9f-8fae-e5a287f65f13 service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:54.499427 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa0e15b-e891-496b-801c-7341bcc8c257 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:54.508399 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 868/1732] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:35:54 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:35:54.531659 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b87d69-5667-452d-9e61-0724b49897eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:54.531659 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b87d69-5667-452d-9e61-0724b49897eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:54.531659 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b87d69-5667-452d-9e61-0724b49897eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:54.531659 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b87d69-5667-452d-9e61-0724b49897eb admin admin] PATH_INFO: `/v3/projects/d1d37ff009244f60954dff36d912eb8e/users/f1f1e30bfb514f6295e993dbb09c957b/roles/4318f86ba9f74cd4a0bef1c311276657` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:54.577509 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b87d69-5667-452d-9e61-0724b49897eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1d37ff009244f60954dff36d912eb8e, user_id=f1f1e30bfb514f6295e993dbb09c957b, role_id=4318f86ba9f74cd4a0bef1c311276657)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:54.578273 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b87d69-5667-452d-9e61-0724b49897eb admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:54.667783 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d146c1-1d9d-4e9f-8fae-e5a287f65f13 service cinder] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:54.672737 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-22d146c1-1d9d-4e9f-8fae-e5a287f65f13 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:54.695887 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 865/1733] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:35:54 2022] PUT /identity/v3/projects/d1d37ff009244f60954dff36d912eb8e/users/f1f1e30bfb514f6295e993dbb09c957b/roles/4318f86ba9f74cd4a0bef1c311276657 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:35:54.701056 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b87d69-5667-452d-9e61-0724b49897eb admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:54.889476 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b87d69-5667-452d-9e61-0724b49897eb admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:54.902191 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf05b161-6217-4f3c-8eee-930cbdc31ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:54.907960 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf05b161-6217-4f3c-8eee-930cbdc31ae1 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:54.913302 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf05b161-6217-4f3c-8eee-930cbdc31ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:54.913890 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf05b161-6217-4f3c-8eee-930cbdc31ae1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:54.914883 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf05b161-6217-4f3c-8eee-930cbdc31ae1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:54.917898 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf05b161-6217-4f3c-8eee-930cbdc31ae1 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:54.940835 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 866/1734] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:35:54 2022] GET /identity/v3/roles => generated 1420 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:35:54.947875 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf05b161-6217-4f3c-8eee-930cbdc31ae1 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:55.050013 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d146c1-1d9d-4e9f-8fae-e5a287f65f13 service cinder] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:55.093388 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf05b161-6217-4f3c-8eee-930cbdc31ae1 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:55.100279 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7887bc-8d32-49aa-99c0-d13e0a601158 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86c2df3d378490ca7ea497a73c19d40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:55.101076 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7887bc-8d32-49aa-99c0-d13e0a601158 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:55.101464 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7887bc-8d32-49aa-99c0-d13e0a601158 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:55.101906 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7887bc-8d32-49aa-99c0-d13e0a601158 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:55.104575 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-8c7887bc-8d32-49aa-99c0-d13e0a601158 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:55.186710 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7887bc-8d32-49aa-99c0-d13e0a601158 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:55.188311 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7887bc-8d32-49aa-99c0-d13e0a601158 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:55.214185 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 867/1735] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:35:54 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:35:55.215160 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7887bc-8d32-49aa-99c0-d13e0a601158 service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:55.227458 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335505c5-6c25-4ec0-93a1-8179c9569ab7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef76f7fe39a3499aaeebf85c176804f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:55.228705 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335505c5-6c25-4ec0-93a1-8179c9569ab7 service swift] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:55.228705 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335505c5-6c25-4ec0-93a1-8179c9569ab7 service swift] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:55.228705 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335505c5-6c25-4ec0-93a1-8179c9569ab7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:55.257036 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7887bc-8d32-49aa-99c0-d13e0a601158 service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:55.271424 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0debb96c-5cc6-49bf-ace4-335645a3efa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:55.272799 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0debb96c-5cc6-49bf-ace4-335645a3efa6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:55.273082 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0debb96c-5cc6-49bf-ace4-335645a3efa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:55.273357 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0debb96c-5cc6-49bf-ace4-335645a3efa6 admin admin] PATH_INFO: `/v3/projects/d1d37ff009244f60954dff36d912eb8e/users/f1f1e30bfb514f6295e993dbb09c957b/roles/4318f86ba9f74cd4a0bef1c311276657` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:55.282786 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0debb96c-5cc6-49bf-ace4-335645a3efa6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1d37ff009244f60954dff36d912eb8e, user_id=f1f1e30bfb514f6295e993dbb09c957b, role_id=4318f86ba9f74cd4a0bef1c311276657)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:55.285548 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0debb96c-5cc6-49bf-ace4-335645a3efa6 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:55.311225 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:35:55.319736 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 868/1736] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:35:55 2022] PUT /identity/v3/projects/d1d37ff009244f60954dff36d912eb8e/users/f1f1e30bfb514f6295e993dbb09c957b/roles/4318f86ba9f74cd4a0bef1c311276657 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:35:55.321373 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8b7414-9fef-4da2-a5d4-198f87de4788 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:55.321644 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8b7414-9fef-4da2-a5d4-198f87de4788 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:55.321928 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8b7414-9fef-4da2-a5d4-198f87de4788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:55.392193 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-fc8b7414-9fef-4da2-a5d4-198f87de4788 None None] MFA Rules not processed for user `f1f1e30bfb514f6295e993dbb09c957b`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:35:55.393321 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-fc8b7414-9fef-4da2-a5d4-198f87de4788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:55.415543 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335505c5-6c25-4ec0-93a1-8179c9569ab7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:55.435782 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335505c5-6c25-4ec0-93a1-8179c9569ab7 service swift] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:55.491251 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 869/1737] 10.222.0.33 () {60 vars in 1384 bytes} [Fri Jul 15 19:35:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 820 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 15 19:35:55.655846 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:35:55.671738 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 869/1738] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:35:55 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:35:55.680180 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335505c5-6c25-4ec0-93a1-8179c9569ab7 service swift] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:55.683147 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-335505c5-6c25-4ec0-93a1-8179c9569ab7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:55.715942 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335505c5-6c25-4ec0-93a1-8179c9569ab7 service swift] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:55.913121 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac547c3-0d3c-4735-bf53-1e20c9c69a10 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86c2df3d378490ca7ea497a73c19d40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:55.914131 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac547c3-0d3c-4735-bf53-1e20c9c69a10 service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:55.914615 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac547c3-0d3c-4735-bf53-1e20c9c69a10 service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:55.915085 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac547c3-0d3c-4735-bf53-1e20c9c69a10 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:55.930363 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac547c3-0d3c-4735-bf53-1e20c9c69a10 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:55.933710 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac547c3-0d3c-4735-bf53-1e20c9c69a10 service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:55.963145 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 870/1739] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:35:55 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:35:56.271774 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8b7414-9fef-4da2-a5d4-198f87de4788 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:56.273476 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-fc8b7414-9fef-4da2-a5d4-198f87de4788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:56.381209 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac547c3-0d3c-4735-bf53-1e20c9c69a10 service cinder] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:56.642646 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8b7414-9fef-4da2-a5d4-198f87de4788 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:56.659309 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac547c3-0d3c-4735-bf53-1e20c9c69a10 service cinder] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:56.662871 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca8caea-9b6e-4abd-83e0-ba1f2a01f769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:56.664074 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca8caea-9b6e-4abd-83e0-ba1f2a01f769 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:56.665365 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca8caea-9b6e-4abd-83e0-ba1f2a01f769 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:56.665833 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca8caea-9b6e-4abd-83e0-ba1f2a01f769 admin admin] PATH_INFO: `/v3/projects/692a06b46d6447898538caea96addcb5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:56.668972 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca8caea-9b6e-4abd-83e0-ba1f2a01f769 admin admin] RBAC: Authorizing `identity:delete_project(project_id=692a06b46d6447898538caea96addcb5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:56.672260 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca8caea-9b6e-4abd-83e0-ba1f2a01f769 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:56.683821 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3909e15-eb7b-4f66-be89-ea71a855dabf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:56.685019 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3909e15-eb7b-4f66-be89-ea71a855dabf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:56.686151 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3909e15-eb7b-4f66-be89-ea71a855dabf admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:56.692126 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3909e15-eb7b-4f66-be89-ea71a855dabf admin admin] PATH_INFO: `/v3/users/77dd6b8d55ae4eddad0b5d8cf0f389c1` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:56.713140 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3909e15-eb7b-4f66-be89-ea71a855dabf admin admin] RBAC: Authorizing `identity:delete_user(user_id=77dd6b8d55ae4eddad0b5d8cf0f389c1)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:56.716654 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3909e15-eb7b-4f66-be89-ea71a855dabf admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:56.921294 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:35:56.962100 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-fca8caea-9b6e-4abd-83e0-ba1f2a01f769 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '692a06b46d6447898538caea96addcb5', 'request_id': 'req-fca8caea-9b6e-4abd-83e0-ba1f2a01f769'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:35:57.055320 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-b3909e15-eb7b-4f66-be89-ea71a855dabf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77dd6b8d55ae4eddad0b5d8cf0f389c1'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:35:57.082339 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-b3909e15-eb7b-4f66-be89-ea71a855dabf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77dd6b8d55ae4eddad0b5d8cf0f389c1'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:35:57.116409 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-fca8caea-9b6e-4abd-83e0-ba1f2a01f769 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '692a06b46d6447898538caea96addcb5', 'request_id': 'req-fca8caea-9b6e-4abd-83e0-ba1f2a01f769'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:35:57.125282 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-fca8caea-9b6e-4abd-83e0-ba1f2a01f769 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '692a06b46d6447898538caea96addcb5', 'request_id': 'req-fca8caea-9b6e-4abd-83e0-ba1f2a01f769'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:35:57.169115 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-fca8caea-9b6e-4abd-83e0-ba1f2a01f769 admin admin] The token cache is being invalidate because project 692a06b46d6447898538caea96addcb5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:35:57.169435 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-fca8caea-9b6e-4abd-83e0-ba1f2a01f769 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:35:57.170875 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-fca8caea-9b6e-4abd-83e0-ba1f2a01f769 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:35:57.182388 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 870/1740] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jul 15 19:35:56 2022] DELETE /identity/v3/projects/692a06b46d6447898538caea96addcb5 => generated 0 bytes in 911 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:35:57.184412 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a768da9-6c47-4b64-8636-bf392d8b9107 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:57.184412 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a768da9-6c47-4b64-8636-bf392d8b9107 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:57.184864 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a768da9-6c47-4b64-8636-bf392d8b9107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:57.261991 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-b3909e15-eb7b-4f66-be89-ea71a855dabf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77dd6b8d55ae4eddad0b5d8cf0f389c1'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:35:57.262642 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-b3909e15-eb7b-4f66-be89-ea71a855dabf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77dd6b8d55ae4eddad0b5d8cf0f389c1'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:35:57.263290 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-b3909e15-eb7b-4f66-be89-ea71a855dabf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77dd6b8d55ae4eddad0b5d8cf0f389c1'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:35:57.263970 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-9a768da9-6c47-4b64-8636-bf392d8b9107 None None] MFA Rules not processed for user `b2a283d7d3594e37943bda4529d0a775`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:35:57.265207 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9a768da9-6c47-4b64-8636-bf392d8b9107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:57.292284 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 871/1741] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jul 15 19:35:56 2022] DELETE /identity/v3/users/77dd6b8d55ae4eddad0b5d8cf0f389c1 => generated 0 bytes in 908 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:35:57.294001 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3af6ef-68ea-4022-88b2-643f5b9127ce None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:57.298585 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3af6ef-68ea-4022-88b2-643f5b9127ce None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:57.298585 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3af6ef-68ea-4022-88b2-643f5b9127ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:57.363967 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-7d3af6ef-68ea-4022-88b2-643f5b9127ce None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:35:57.373284 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:35:57.374848 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-7d3af6ef-68ea-4022-88b2-643f5b9127ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:57.436886 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 871/1742] 10.222.0.30 () {58 vars in 1015 bytes} [Fri Jul 15 19:35:57 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:35:57.445460 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c6a5d8-a028-463d-b9ed-b0fe2d5c1957 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:57.445787 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c6a5d8-a028-463d-b9ed-b0fe2d5c1957 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:57.448634 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c6a5d8-a028-463d-b9ed-b0fe2d5c1957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:57.537217 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-52c6a5d8-a028-463d-b9ed-b0fe2d5c1957 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:35:57.538500 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-52c6a5d8-a028-463d-b9ed-b0fe2d5c1957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:57.686172 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 872/1743] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:35:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:35:57.686172 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3af6ef-68ea-4022-88b2-643f5b9127ce None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:57.686172 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-7d3af6ef-68ea-4022-88b2-643f5b9127ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:57.688006 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 872/1744] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:35:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:35:57.693040 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef17b9c4-18f1-4c23-b508-88bd538e35e9 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:57.693330 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef17b9c4-18f1-4c23-b508-88bd538e35e9 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:57.693609 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef17b9c4-18f1-4c23-b508-88bd538e35e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:57.750936 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-ef17b9c4-18f1-4c23-b508-88bd538e35e9 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:35:57.753499 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-ef17b9c4-18f1-4c23-b508-88bd538e35e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:57.810769 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3af6ef-68ea-4022-88b2-643f5b9127ce None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:57.810769 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e27624-10de-4cec-ad3d-c85f79b8586b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:57.810769 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e27624-10de-4cec-ad3d-c85f79b8586b service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:57.810769 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e27624-10de-4cec-ad3d-c85f79b8586b service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:57.812365 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e27624-10de-4cec-ad3d-c85f79b8586b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:57.820345 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e27624-10de-4cec-ad3d-c85f79b8586b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:57.822448 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e27624-10de-4cec-ad3d-c85f79b8586b service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:57.835118 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 873/1745] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:35:57 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:35:57.836462 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e27624-10de-4cec-ad3d-c85f79b8586b service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:57.837343 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 873/1746] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:35:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:35:57.843233 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef17b9c4-18f1-4c23-b508-88bd538e35e9 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:57.846548 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e27624-10de-4cec-ad3d-c85f79b8586b service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:57.854605 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79036b65-27b7-4ca4-abc6-e587ba223dc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:57.856000 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79036b65-27b7-4ca4-abc6-e587ba223dc4 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:57.857750 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79036b65-27b7-4ca4-abc6-e587ba223dc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:57.859188 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79036b65-27b7-4ca4-abc6-e587ba223dc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:57.883737 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79036b65-27b7-4ca4-abc6-e587ba223dc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:57.889421 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79036b65-27b7-4ca4-abc6-e587ba223dc4 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:57.893106 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef17b9c4-18f1-4c23-b508-88bd538e35e9 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:57.917806 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28323d51-204f-4509-8064-3125b5cfcf54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:57.919065 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28323d51-204f-4509-8064-3125b5cfcf54 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:57.919805 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28323d51-204f-4509-8064-3125b5cfcf54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:57.920359 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28323d51-204f-4509-8064-3125b5cfcf54 admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:57.924919 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 874/1747] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:35:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:35:57.925336 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28323d51-204f-4509-8064-3125b5cfcf54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:35:57.926000 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28323d51-204f-4509-8064-3125b5cfcf54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:57.928251 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28323d51-204f-4509-8064-3125b5cfcf54 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:57.963142 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 874/1748] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jul 15 19:35:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:35:57.967289 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79036b65-27b7-4ca4-abc6-e587ba223dc4 service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:57.979964 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79036b65-27b7-4ca4-abc6-e587ba223dc4 service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:58.010649 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34dcb21-777c-4a1d-843a-056737bcb6ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:58.011566 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34dcb21-777c-4a1d-843a-056737bcb6ad admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:58.012264 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34dcb21-777c-4a1d-843a-056737bcb6ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:58.012603 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34dcb21-777c-4a1d-843a-056737bcb6ad admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:58.013511 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34dcb21-777c-4a1d-843a-056737bcb6ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:58.015878 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34dcb21-777c-4a1d-843a-056737bcb6ad admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:58.189083 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 875/1749] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jul 15 19:35:57 2022] POST /identity/v3/projects => generated 381 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:35:58.194826 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28323d51-204f-4509-8064-3125b5cfcf54 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:58.217617 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28323d51-204f-4509-8064-3125b5cfcf54 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:58.457190 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff109bd5-cdb5-41eb-9fe4-4f00f6e47f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:58.461162 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff109bd5-cdb5-41eb-9fe4-4f00f6e47f33 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:58.461905 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff109bd5-cdb5-41eb-9fe4-4f00f6e47f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:58.462491 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff109bd5-cdb5-41eb-9fe4-4f00f6e47f33 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:58.463662 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff109bd5-cdb5-41eb-9fe4-4f00f6e47f33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:58.494146 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff109bd5-cdb5-41eb-9fe4-4f00f6e47f33 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:58.611990 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 875/1750] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jul 15 19:35:58 2022] POST /identity/v3/users => generated 291 bytes in 418 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:35:58.618626 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34dcb21-777c-4a1d-843a-056737bcb6ad admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:58.647780 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34dcb21-777c-4a1d-843a-056737bcb6ad admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:58.659375 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df0bc60-94eb-4fed-b084-f260070d93cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:58.660226 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df0bc60-94eb-4fed-b084-f260070d93cb admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:58.662254 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df0bc60-94eb-4fed-b084-f260070d93cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:58.662934 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df0bc60-94eb-4fed-b084-f260070d93cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:58.664443 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df0bc60-94eb-4fed-b084-f260070d93cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:58.667140 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df0bc60-94eb-4fed-b084-f260070d93cb admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:58.686778 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 876/1751] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:35:58 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:35:58.691978 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff109bd5-cdb5-41eb-9fe4-4f00f6e47f33 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:58.709478 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:35:58.722285 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff109bd5-cdb5-41eb-9fe4-4f00f6e47f33 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:58.742458 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaa1221-8d13-4f56-95e9-a55f4e354ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:58.746803 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaa1221-8d13-4f56-95e9-a55f4e354ac3 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:58.747398 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaa1221-8d13-4f56-95e9-a55f4e354ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:58.748682 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaa1221-8d13-4f56-95e9-a55f4e354ac3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:58.755473 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaa1221-8d13-4f56-95e9-a55f4e354ac3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:58.764361 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaa1221-8d13-4f56-95e9-a55f4e354ac3 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:58.828894 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df0bc60-94eb-4fed-b084-f260070d93cb admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:58.844797 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 876/1752] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:35:58 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:35:58.853006 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df0bc60-94eb-4fed-b084-f260070d93cb admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:58.887593 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e6d051-9e40-43b6-a90a-f006282b6540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:58.893274 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e6d051-9e40-43b6-a90a-f006282b6540 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:58.894444 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e6d051-9e40-43b6-a90a-f006282b6540 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:58.897811 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e6d051-9e40-43b6-a90a-f006282b6540 admin admin] PATH_INFO: `/v3/projects/3b4769ea3e1445328fade580459a8ff4/users/e50265cf1f244d62ac5e46117d6bce82/roles/28763c6e70094116b3a4b2895895fa58` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:58.954614 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e6d051-9e40-43b6-a90a-f006282b6540 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b4769ea3e1445328fade580459a8ff4, user_id=e50265cf1f244d62ac5e46117d6bce82, role_id=28763c6e70094116b3a4b2895895fa58)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:58.960493 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e6d051-9e40-43b6-a90a-f006282b6540 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:59.079945 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 877/1753] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:35:58 2022] PUT /identity/v3/projects/3b4769ea3e1445328fade580459a8ff4/users/e50265cf1f244d62ac5e46117d6bce82/roles/28763c6e70094116b3a4b2895895fa58 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:35:59.093401 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592def61-7287-41de-8273-4563f40b8da0 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:59.095332 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592def61-7287-41de-8273-4563f40b8da0 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:59.095768 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592def61-7287-41de-8273-4563f40b8da0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:59.183556 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-592def61-7287-41de-8273-4563f40b8da0 None None] MFA Rules not processed for user `e50265cf1f244d62ac5e46117d6bce82`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:35:59.188938 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-592def61-7287-41de-8273-4563f40b8da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:59.301508 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 877/1754] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:35:59 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:35:59.306700 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e6d051-9e40-43b6-a90a-f006282b6540 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:59.477189 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e6d051-9e40-43b6-a90a-f006282b6540 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:59.487794 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7554ef18-c59e-4efe-bc62-9b31975e5c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:59.489530 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7554ef18-c59e-4efe-bc62-9b31975e5c78 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:59.490126 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7554ef18-c59e-4efe-bc62-9b31975e5c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:59.490822 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7554ef18-c59e-4efe-bc62-9b31975e5c78 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:59.492267 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7554ef18-c59e-4efe-bc62-9b31975e5c78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:59.496219 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7554ef18-c59e-4efe-bc62-9b31975e5c78 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:59.621833 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 878/1755] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jul 15 19:35:59 2022] POST /identity/v3/projects => generated 381 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:35:59.625797 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592def61-7287-41de-8273-4563f40b8da0 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:59.641388 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592def61-7287-41de-8273-4563f40b8da0 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:59.750203 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:35:59.792874 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e1f293-2f3f-481b-ac24-21a24cb725a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:59.792874 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e1f293-2f3f-481b-ac24-21a24cb725a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:59.793178 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e1f293-2f3f-481b-ac24-21a24cb725a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:59.793552 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e1f293-2f3f-481b-ac24-21a24cb725a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:59.794299 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e1f293-2f3f-481b-ac24-21a24cb725a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:59.801149 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e1f293-2f3f-481b-ac24-21a24cb725a9 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:59.937182 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 878/1756] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jul 15 19:35:59 2022] POST /identity/v3/users => generated 297 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:35:59.942080 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7554ef18-c59e-4efe-bc62-9b31975e5c78 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:00.007629 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7554ef18-c59e-4efe-bc62-9b31975e5c78 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:00.031573 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce1b697-74b9-4411-918e-c8588e47ffbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:00.043675 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce1b697-74b9-4411-918e-c8588e47ffbb admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:00.046443 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce1b697-74b9-4411-918e-c8588e47ffbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:00.047299 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce1b697-74b9-4411-918e-c8588e47ffbb admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:00.052340 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce1b697-74b9-4411-918e-c8588e47ffbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:00.060615 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce1b697-74b9-4411-918e-c8588e47ffbb admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:00.121601 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 879/1757] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:35:59 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:36:00.126192 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e1f293-2f3f-481b-ac24-21a24cb725a9 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:00.177137 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e1f293-2f3f-481b-ac24-21a24cb725a9 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:00.221834 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d1057a-e4a9-446b-93d3-a882bf0145cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:00.228705 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d1057a-e4a9-446b-93d3-a882bf0145cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:00.229880 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d1057a-e4a9-446b-93d3-a882bf0145cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:00.231954 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d1057a-e4a9-446b-93d3-a882bf0145cf admin admin] PATH_INFO: `/v3/projects/32c5433b12fd49cfb776e256cd7f6fa3/users/9001a5ff5fd84f35b1cf5a634b264e30/roles/4318f86ba9f74cd4a0bef1c311276657` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:00.341871 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d1057a-e4a9-446b-93d3-a882bf0145cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=32c5433b12fd49cfb776e256cd7f6fa3, user_id=9001a5ff5fd84f35b1cf5a634b264e30, role_id=4318f86ba9f74cd4a0bef1c311276657)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:00.355067 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d1057a-e4a9-446b-93d3-a882bf0145cf admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:00.608781 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 879/1758] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:36:00 2022] PUT /identity/v3/projects/32c5433b12fd49cfb776e256cd7f6fa3/users/9001a5ff5fd84f35b1cf5a634b264e30/roles/4318f86ba9f74cd4a0bef1c311276657 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:36:00.609935 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce1b697-74b9-4411-918e-c8588e47ffbb admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:00.635270 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce1b697-74b9-4411-918e-c8588e47ffbb admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:00.758776 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:36:00.900208 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188b9fdc-7492-4386-8836-bec5fb71495c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:00.901444 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188b9fdc-7492-4386-8836-bec5fb71495c admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:00.902881 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188b9fdc-7492-4386-8836-bec5fb71495c admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:00.903273 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188b9fdc-7492-4386-8836-bec5fb71495c admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:00.907727 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188b9fdc-7492-4386-8836-bec5fb71495c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:00.912187 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188b9fdc-7492-4386-8836-bec5fb71495c admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:00.930738 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:36:00.944565 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 880/1759] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:36:00 2022] GET /identity/v3/roles => generated 1420 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:36:00.949009 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d1057a-e4a9-446b-93d3-a882bf0145cf admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:00.965616 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d1057a-e4a9-446b-93d3-a882bf0145cf admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:00.975958 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af33d9c-632a-4aa5-beed-4069bc2a93e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:00.977154 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af33d9c-632a-4aa5-beed-4069bc2a93e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:00.977679 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af33d9c-632a-4aa5-beed-4069bc2a93e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:00.978218 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af33d9c-632a-4aa5-beed-4069bc2a93e7 admin admin] PATH_INFO: `/v3/projects/32c5433b12fd49cfb776e256cd7f6fa3/users/9001a5ff5fd84f35b1cf5a634b264e30/roles/4318f86ba9f74cd4a0bef1c311276657` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:00.980692 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af33d9c-632a-4aa5-beed-4069bc2a93e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32c5433b12fd49cfb776e256cd7f6fa3, user_id=9001a5ff5fd84f35b1cf5a634b264e30, role_id=4318f86ba9f74cd4a0bef1c311276657)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:00.991084 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af33d9c-632a-4aa5-beed-4069bc2a93e7 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:01.010788 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 880/1760] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:36:00 2022] PUT /identity/v3/projects/32c5433b12fd49cfb776e256cd7f6fa3/users/9001a5ff5fd84f35b1cf5a634b264e30/roles/4318f86ba9f74cd4a0bef1c311276657 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:36:01.022594 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880bbab3-fd06-464b-89e4-695d1a149d6b None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:01.023919 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880bbab3-fd06-464b-89e4-695d1a149d6b None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:01.024717 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880bbab3-fd06-464b-89e4-695d1a149d6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:01.144229 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-880bbab3-fd06-464b-89e4-695d1a149d6b None None] MFA Rules not processed for user `9001a5ff5fd84f35b1cf5a634b264e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:36:01.154280 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-880bbab3-fd06-464b-89e4-695d1a149d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:36:01.335120 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af33d9c-632a-4aa5-beed-4069bc2a93e7 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:01.338581 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-7af33d9c-632a-4aa5-beed-4069bc2a93e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:36:01.398202 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:36:01.403442 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 881/1761] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:36:01 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 383 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:36:01.418038 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880bbab3-fd06-464b-89e4-695d1a149d6b None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:01.419986 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-880bbab3-fd06-464b-89e4-695d1a149d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:36:01.551457 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af33d9c-632a-4aa5-beed-4069bc2a93e7 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:01.585450 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880bbab3-fd06-464b-89e4-695d1a149d6b None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:01.588354 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3bd3c3-7b92-4578-b68e-c22fc1fdf534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:01.591016 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3bd3c3-7b92-4578-b68e-c22fc1fdf534 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:01.592190 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3bd3c3-7b92-4578-b68e-c22fc1fdf534 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:01.595101 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3bd3c3-7b92-4578-b68e-c22fc1fdf534 admin admin] PATH_INFO: `/v3/projects/43ec1f1f67f84fdf96cb309e823df761` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:01.599084 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96a81f0-5dcb-45fa-a7f2-5d93cad98397 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86c2df3d378490ca7ea497a73c19d40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:01.599780 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96a81f0-5dcb-45fa-a7f2-5d93cad98397 service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:01.602470 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96a81f0-5dcb-45fa-a7f2-5d93cad98397 service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:01.602470 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96a81f0-5dcb-45fa-a7f2-5d93cad98397 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:01.602652 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3bd3c3-7b92-4578-b68e-c22fc1fdf534 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43ec1f1f67f84fdf96cb309e823df761)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:01.608355 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3bd3c3-7b92-4578-b68e-c22fc1fdf534 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:01.640723 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96a81f0-5dcb-45fa-a7f2-5d93cad98397 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:01.642642 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96a81f0-5dcb-45fa-a7f2-5d93cad98397 service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:01.686424 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 882/1762] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:36:01 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:36:01.861587 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ed3bd3c3-7b92-4578-b68e-c22fc1fdf534 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43ec1f1f67f84fdf96cb309e823df761', 'request_id': 'req-ed3bd3c3-7b92-4578-b68e-c22fc1fdf534'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:01.948512 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ed3bd3c3-7b92-4578-b68e-c22fc1fdf534 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43ec1f1f67f84fdf96cb309e823df761', 'request_id': 'req-ed3bd3c3-7b92-4578-b68e-c22fc1fdf534'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:01.956155 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ed3bd3c3-7b92-4578-b68e-c22fc1fdf534 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43ec1f1f67f84fdf96cb309e823df761', 'request_id': 'req-ed3bd3c3-7b92-4578-b68e-c22fc1fdf534'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:02.022239 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ed3bd3c3-7b92-4578-b68e-c22fc1fdf534 admin admin] The token cache is being invalidate because project 43ec1f1f67f84fdf96cb309e823df761 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:36:02.022679 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ed3bd3c3-7b92-4578-b68e-c22fc1fdf534 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:02.024269 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ed3bd3c3-7b92-4578-b68e-c22fc1fdf534 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:02.048153 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 881/1763] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jul 15 19:36:01 2022] DELETE /identity/v3/projects/43ec1f1f67f84fdf96cb309e823df761 => generated 0 bytes in 714 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:36:02.052374 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96a81f0-5dcb-45fa-a7f2-5d93cad98397 service cinder] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:02.068612 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96a81f0-5dcb-45fa-a7f2-5d93cad98397 service cinder] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:02.262106 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1a4533-c804-49d3-bcc5-35c6d2589993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:02.263011 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1a4533-c804-49d3-bcc5-35c6d2589993 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:02.263296 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1a4533-c804-49d3-bcc5-35c6d2589993 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:02.263572 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1a4533-c804-49d3-bcc5-35c6d2589993 admin admin] PATH_INFO: `/v3/users/f1f1e30bfb514f6295e993dbb09c957b` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:02.272580 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1a4533-c804-49d3-bcc5-35c6d2589993 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1f1e30bfb514f6295e993dbb09c957b)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:02.274708 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:36:02.285758 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1a4533-c804-49d3-bcc5-35c6d2589993 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:02.649248 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-cd1a4533-c804-49d3-bcc5-35c6d2589993 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1f1e30bfb514f6295e993dbb09c957b'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:02.694761 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-cd1a4533-c804-49d3-bcc5-35c6d2589993 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1f1e30bfb514f6295e993dbb09c957b'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:02.858172 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-cd1a4533-c804-49d3-bcc5-35c6d2589993 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1f1e30bfb514f6295e993dbb09c957b'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:02.858172 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-cd1a4533-c804-49d3-bcc5-35c6d2589993 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1f1e30bfb514f6295e993dbb09c957b'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:02.858172 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-cd1a4533-c804-49d3-bcc5-35c6d2589993 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1f1e30bfb514f6295e993dbb09c957b'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:02.885962 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 883/1764] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jul 15 19:36:02 2022] DELETE /identity/v3/users/f1f1e30bfb514f6295e993dbb09c957b => generated 0 bytes in 834 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:36:03.644696 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3bd3c3-7b92-4578-b68e-c22fc1fdf534 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:03.649151 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-ed3bd3c3-7b92-4578-b68e-c22fc1fdf534 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:36:03.775345 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:36:03.816196 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03365d5-191c-4418-9ab4-2865c79c485e None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:03.818268 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03365d5-191c-4418-9ab4-2865c79c485e None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:03.818566 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03365d5-191c-4418-9ab4-2865c79c485e None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:03.820893 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 884/1765] 10.222.0.33 () {54 vars in 921 bytes} [Fri Jul 15 19:36:03 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:36:03.875376 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3bd3c3-7b92-4578-b68e-c22fc1fdf534 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:03.982206 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5256ed-1ac0-4c2b-be40-a2942d93e8a7 tempest-VolumeQuotaClassesTest-1040707497 tempest-VolumeQuotaClassesTest-1040707497-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9001a5ff5fd84f35b1cf5a634b264e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32c5433b12fd49cfb776e256cd7f6fa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:03.983069 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5256ed-1ac0-4c2b-be40-a2942d93e8a7 tempest-VolumeQuotaClassesTest-1040707497 tempest-VolumeQuotaClassesTest-1040707497-project-admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:03.984396 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5256ed-1ac0-4c2b-be40-a2942d93e8a7 tempest-VolumeQuotaClassesTest-1040707497 tempest-VolumeQuotaClassesTest-1040707497-project-admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:03.988402 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5256ed-1ac0-4c2b-be40-a2942d93e8a7 tempest-VolumeQuotaClassesTest-1040707497 tempest-VolumeQuotaClassesTest-1040707497-project-admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:03.988402 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5256ed-1ac0-4c2b-be40-a2942d93e8a7 tempest-VolumeQuotaClassesTest-1040707497 tempest-VolumeQuotaClassesTest-1040707497-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:36:03.988402 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5256ed-1ac0-4c2b-be40-a2942d93e8a7 tempest-VolumeQuotaClassesTest-1040707497 tempest-VolumeQuotaClassesTest-1040707497-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:03.997809 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5256ed-1ac0-4c2b-be40-a2942d93e8a7 tempest-VolumeQuotaClassesTest-1040707497 tempest-VolumeQuotaClassesTest-1040707497-project-admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:04.016083 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aabde3b-3887-4611-ad1d-156dfe12305e None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:04.016719 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aabde3b-3887-4611-ad1d-156dfe12305e None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:04.017310 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aabde3b-3887-4611-ad1d-156dfe12305e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:04.078687 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 882/1766] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jul 15 19:36:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 435 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:36:04.081408 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5256ed-1ac0-4c2b-be40-a2942d93e8a7 tempest-VolumeQuotaClassesTest-1040707497 tempest-VolumeQuotaClassesTest-1040707497-project-admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:04.091151 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-5b5256ed-1ac0-4c2b-be40-a2942d93e8a7 tempest-VolumeQuotaClassesTest-1040707497 tempest-VolumeQuotaClassesTest-1040707497-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:36:04.108813 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-7aabde3b-3887-4611-ad1d-156dfe12305e None None] MFA Rules not processed for user `7673a3b5c0324e07a9535ac70ed7a491`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:36:04.114038 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-7aabde3b-3887-4611-ad1d-156dfe12305e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:36:04.138701 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5256ed-1ac0-4c2b-be40-a2942d93e8a7 tempest-VolumeQuotaClassesTest-1040707497 tempest-VolumeQuotaClassesTest-1040707497-project-admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:04.167537 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af1f5dc-8ea6-4e78-a43b-6989452933c8 tempest-VolumeQuotaClassesTest-1040707497 tempest-VolumeQuotaClassesTest-1040707497-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9001a5ff5fd84f35b1cf5a634b264e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32c5433b12fd49cfb776e256cd7f6fa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:04.168863 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af1f5dc-8ea6-4e78-a43b-6989452933c8 tempest-VolumeQuotaClassesTest-1040707497 tempest-VolumeQuotaClassesTest-1040707497-project-admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:04.169308 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af1f5dc-8ea6-4e78-a43b-6989452933c8 tempest-VolumeQuotaClassesTest-1040707497 tempest-VolumeQuotaClassesTest-1040707497-project-admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:04.169880 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af1f5dc-8ea6-4e78-a43b-6989452933c8 tempest-VolumeQuotaClassesTest-1040707497 tempest-VolumeQuotaClassesTest-1040707497-project-admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:04.171217 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af1f5dc-8ea6-4e78-a43b-6989452933c8 tempest-VolumeQuotaClassesTest-1040707497 tempest-VolumeQuotaClassesTest-1040707497-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:04.199014 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af1f5dc-8ea6-4e78-a43b-6989452933c8 tempest-VolumeQuotaClassesTest-1040707497 tempest-VolumeQuotaClassesTest-1040707497-project-admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:04.218520 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:36:04.409253 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 885/1767] 10.222.0.33 () {58 vars in 1018 bytes} [Fri Jul 15 19:36:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 395 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:36:04.417938 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b7dda6-d696-4cf8-9939-3c262716f000 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:04.418430 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b7dda6-d696-4cf8-9939-3c262716f000 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:04.418958 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b7dda6-d696-4cf8-9939-3c262716f000 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:04.447730 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 886/1768] 10.222.0.33 () {54 vars in 921 bytes} [Fri Jul 15 19:36:04 2022] GET /identity => generated 270 bytes in 38 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:36:04.484011 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 883/1769] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jul 15 19:36:04 2022] POST /identity/v3/projects => generated 355 bytes in 405 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:36:04.491258 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b7dda6-d696-4cf8-9939-3c262716f000 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:04.562115 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960faca2-f9a2-4b85-b73d-522a9dc11e23 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:04.563000 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960faca2-f9a2-4b85-b73d-522a9dc11e23 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:04.563660 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960faca2-f9a2-4b85-b73d-522a9dc11e23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:04.667819 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-960faca2-f9a2-4b85-b73d-522a9dc11e23 None None] MFA Rules not processed for user `7673a3b5c0324e07a9535ac70ed7a491`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:36:04.670029 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-960faca2-f9a2-4b85-b73d-522a9dc11e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:36:04.731867 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b7dda6-d696-4cf8-9939-3c262716f000 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:04.745188 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd15d7e4-a451-444d-91bd-b91bfecb1ddb tempest-VolumeQuotaClassesTest-1040707497 tempest-VolumeQuotaClassesTest-1040707497-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9001a5ff5fd84f35b1cf5a634b264e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32c5433b12fd49cfb776e256cd7f6fa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:04.748430 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd15d7e4-a451-444d-91bd-b91bfecb1ddb tempest-VolumeQuotaClassesTest-1040707497 tempest-VolumeQuotaClassesTest-1040707497-project-admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:04.749563 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd15d7e4-a451-444d-91bd-b91bfecb1ddb tempest-VolumeQuotaClassesTest-1040707497 tempest-VolumeQuotaClassesTest-1040707497-project-admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:04.750976 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd15d7e4-a451-444d-91bd-b91bfecb1ddb tempest-VolumeQuotaClassesTest-1040707497 tempest-VolumeQuotaClassesTest-1040707497-project-admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:04.751840 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd15d7e4-a451-444d-91bd-b91bfecb1ddb tempest-VolumeQuotaClassesTest-1040707497 tempest-VolumeQuotaClassesTest-1040707497-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:36:04.752162 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd15d7e4-a451-444d-91bd-b91bfecb1ddb tempest-VolumeQuotaClassesTest-1040707497 tempest-VolumeQuotaClassesTest-1040707497-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:04.757526 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd15d7e4-a451-444d-91bd-b91bfecb1ddb tempest-VolumeQuotaClassesTest-1040707497 tempest-VolumeQuotaClassesTest-1040707497-project-admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:04.787875 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 887/1770] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jul 15 19:36:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:36:04.883869 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd15d7e4-a451-444d-91bd-b91bfecb1ddb tempest-VolumeQuotaClassesTest-1040707497 tempest-VolumeQuotaClassesTest-1040707497-project-admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:04.898023 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd15d7e4-a451-444d-91bd-b91bfecb1ddb tempest-VolumeQuotaClassesTest-1040707497 tempest-VolumeQuotaClassesTest-1040707497-project-admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:04.911272 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d96437-a333-460d-922e-1bc5769a41c9 tempest-VolumeQuotaClassesTest-1040707497 tempest-VolumeQuotaClassesTest-1040707497-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9001a5ff5fd84f35b1cf5a634b264e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32c5433b12fd49cfb776e256cd7f6fa3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:04.911272 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d96437-a333-460d-922e-1bc5769a41c9 tempest-VolumeQuotaClassesTest-1040707497 tempest-VolumeQuotaClassesTest-1040707497-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:04.911272 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d96437-a333-460d-922e-1bc5769a41c9 tempest-VolumeQuotaClassesTest-1040707497 tempest-VolumeQuotaClassesTest-1040707497-project-admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:04.911984 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d96437-a333-460d-922e-1bc5769a41c9 tempest-VolumeQuotaClassesTest-1040707497 tempest-VolumeQuotaClassesTest-1040707497-project-admin] PATH_INFO: `/v3/projects/2e46080c6f2e43f3b4354ea87c780e5f` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:04.912601 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d96437-a333-460d-922e-1bc5769a41c9 tempest-VolumeQuotaClassesTest-1040707497 tempest-VolumeQuotaClassesTest-1040707497-project-admin] RBAC: Authorizing `identity:delete_project(project_id=2e46080c6f2e43f3b4354ea87c780e5f)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:04.915045 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d96437-a333-460d-922e-1bc5769a41c9 tempest-VolumeQuotaClassesTest-1040707497 tempest-VolumeQuotaClassesTest-1040707497-project-admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:04.931045 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 884/1771] 10.222.0.33 () {58 vars in 1018 bytes} [Fri Jul 15 19:36:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 372 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:36:05.063018 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:36:05.112560 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-f3d96437-a333-460d-922e-1bc5769a41c9 tempest-VolumeQuotaClassesTest-1040707497 tempest-VolumeQuotaClassesTest-1040707497-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e46080c6f2e43f3b4354ea87c780e5f', 'request_id': 'req-f3d96437-a333-460d-922e-1bc5769a41c9'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:05.177250 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960faca2-f9a2-4b85-b73d-522a9dc11e23 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:05.179610 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-960faca2-f9a2-4b85-b73d-522a9dc11e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:36:05.196815 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960faca2-f9a2-4b85-b73d-522a9dc11e23 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:05.209805 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-f3d96437-a333-460d-922e-1bc5769a41c9 tempest-VolumeQuotaClassesTest-1040707497 tempest-VolumeQuotaClassesTest-1040707497-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e46080c6f2e43f3b4354ea87c780e5f', 'request_id': 'req-f3d96437-a333-460d-922e-1bc5769a41c9'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:05.224439 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-f3d96437-a333-460d-922e-1bc5769a41c9 tempest-VolumeQuotaClassesTest-1040707497 tempest-VolumeQuotaClassesTest-1040707497-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e46080c6f2e43f3b4354ea87c780e5f', 'request_id': 'req-f3d96437-a333-460d-922e-1bc5769a41c9'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:05.266420 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-f3d96437-a333-460d-922e-1bc5769a41c9 tempest-VolumeQuotaClassesTest-1040707497 tempest-VolumeQuotaClassesTest-1040707497-project-admin] The token cache is being invalidate because project 2e46080c6f2e43f3b4354ea87c780e5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:36:05.267179 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-f3d96437-a333-460d-922e-1bc5769a41c9 tempest-VolumeQuotaClassesTest-1040707497 tempest-VolumeQuotaClassesTest-1040707497-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:05.267691 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-f3d96437-a333-460d-922e-1bc5769a41c9 tempest-VolumeQuotaClassesTest-1040707497 tempest-VolumeQuotaClassesTest-1040707497-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:05.272632 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 888/1772] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jul 15 19:36:04 2022] DELETE /identity/v3/projects/2e46080c6f2e43f3b4354ea87c780e5f => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:36:05.306596 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7585a998-7886-4e63-8dfc-179dc65e5e88 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7673a3b5c0324e07a9535ac70ed7a491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:05.307645 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7585a998-7886-4e63-8dfc-179dc65e5e88 service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:05.310404 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7585a998-7886-4e63-8dfc-179dc65e5e88 service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:05.310778 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7585a998-7886-4e63-8dfc-179dc65e5e88 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:05.316133 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-7585a998-7886-4e63-8dfc-179dc65e5e88 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:36:05.393161 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7585a998-7886-4e63-8dfc-179dc65e5e88 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:05.395912 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7585a998-7886-4e63-8dfc-179dc65e5e88 service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:05.447869 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 885/1773] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:36:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 19:36:05.652415 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d96437-a333-460d-922e-1bc5769a41c9 tempest-VolumeQuotaClassesTest-1040707497 tempest-VolumeQuotaClassesTest-1040707497-project-admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:05.657839 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-f3d96437-a333-460d-922e-1bc5769a41c9 tempest-VolumeQuotaClassesTest-1040707497 tempest-VolumeQuotaClassesTest-1040707497-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:36:05.710398 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d96437-a333-460d-922e-1bc5769a41c9 tempest-VolumeQuotaClassesTest-1040707497 tempest-VolumeQuotaClassesTest-1040707497-project-admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:05.722581 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7585a998-7886-4e63-8dfc-179dc65e5e88 service nova] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:05.724827 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-7585a998-7886-4e63-8dfc-179dc65e5e88 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:36:05.730776 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f0fff5-205f-4511-8071-6c7cc4788488 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7673a3b5c0324e07a9535ac70ed7a491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:05.731320 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f0fff5-205f-4511-8071-6c7cc4788488 service nova] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:05.731606 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f0fff5-205f-4511-8071-6c7cc4788488 service nova] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:05.731881 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f0fff5-205f-4511-8071-6c7cc4788488 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:05.738693 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-a5f0fff5-205f-4511-8071-6c7cc4788488 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:36:05.766499 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f0fff5-205f-4511-8071-6c7cc4788488 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:05.768157 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f0fff5-205f-4511-8071-6c7cc4788488 service nova] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:05.801357 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 889/1774] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:36:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:36:05.925918 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7585a998-7886-4e63-8dfc-179dc65e5e88 service nova] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:05.940923 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d4b66e-03e8-4767-84ac-eb1b5dd19814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:05.943135 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4b66e-03e8-4767-84ac-eb1b5dd19814 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:05.943626 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4b66e-03e8-4767-84ac-eb1b5dd19814 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:05.944365 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4b66e-03e8-4767-84ac-eb1b5dd19814 admin admin] PATH_INFO: `/v3/projects/d1d37ff009244f60954dff36d912eb8e` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:05.946172 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d4b66e-03e8-4767-84ac-eb1b5dd19814 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1d37ff009244f60954dff36d912eb8e)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:05.948480 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d4b66e-03e8-4767-84ac-eb1b5dd19814 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:06.157008 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b9d4b66e-03e8-4767-84ac-eb1b5dd19814 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1d37ff009244f60954dff36d912eb8e', 'request_id': 'req-b9d4b66e-03e8-4767-84ac-eb1b5dd19814'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:06.245119 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b9d4b66e-03e8-4767-84ac-eb1b5dd19814 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1d37ff009244f60954dff36d912eb8e', 'request_id': 'req-b9d4b66e-03e8-4767-84ac-eb1b5dd19814'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:06.266480 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b9d4b66e-03e8-4767-84ac-eb1b5dd19814 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1d37ff009244f60954dff36d912eb8e', 'request_id': 'req-b9d4b66e-03e8-4767-84ac-eb1b5dd19814'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:06.313124 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b9d4b66e-03e8-4767-84ac-eb1b5dd19814 admin admin] The token cache is being invalidate because project d1d37ff009244f60954dff36d912eb8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:36:06.313988 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b9d4b66e-03e8-4767-84ac-eb1b5dd19814 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:06.315103 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b9d4b66e-03e8-4767-84ac-eb1b5dd19814 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:06.317854 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 886/1775] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jul 15 19:36:05 2022] DELETE /identity/v3/projects/d1d37ff009244f60954dff36d912eb8e => generated 0 bytes in 596 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:36:06.323015 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f48ef95-eac9-4263-8b0d-619937a630a1 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:06.323315 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f48ef95-eac9-4263-8b0d-619937a630a1 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:06.323682 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f48ef95-eac9-4263-8b0d-619937a630a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:06.351198 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-7f48ef95-eac9-4263-8b0d-619937a630a1 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:36:06.352563 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-7f48ef95-eac9-4263-8b0d-619937a630a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:36:06.498222 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 890/1776] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:36:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:36:06.500145 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb52526-7b9d-45fe-b261-598b3a212d37 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:06.500464 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb52526-7b9d-45fe-b261-598b3a212d37 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:06.500786 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb52526-7b9d-45fe-b261-598b3a212d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:06.555007 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-bdb52526-7b9d-45fe-b261-598b3a212d37 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:36:06.563250 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-bdb52526-7b9d-45fe-b261-598b3a212d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:36:06.608103 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 887/1777] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:36:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:36:06.612350 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f48ef95-eac9-4263-8b0d-619937a630a1 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:06.630565 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb52526-7b9d-45fe-b261-598b3a212d37 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:06.640707 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-bdb52526-7b9d-45fe-b261-598b3a212d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:36:06.661167 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f48ef95-eac9-4263-8b0d-619937a630a1 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:06.699129 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8f59db-74c1-4a34-ac38-f1765756a95f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:06.703226 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8f59db-74c1-4a34-ac38-f1765756a95f admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:06.703386 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8f59db-74c1-4a34-ac38-f1765756a95f admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:06.703651 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8f59db-74c1-4a34-ac38-f1765756a95f admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:06.714603 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8f59db-74c1-4a34-ac38-f1765756a95f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:36:06.714603 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8f59db-74c1-4a34-ac38-f1765756a95f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:06.720016 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8f59db-74c1-4a34-ac38-f1765756a95f admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:06.736875 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:36:06.742851 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb52526-7b9d-45fe-b261-598b3a212d37 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:06.763070 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef5c9c4-cdd7-42cf-bb0d-94d3cb09a095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:06.767449 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef5c9c4-cdd7-42cf-bb0d-94d3cb09a095 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:06.769543 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef5c9c4-cdd7-42cf-bb0d-94d3cb09a095 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:06.773500 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef5c9c4-cdd7-42cf-bb0d-94d3cb09a095 admin admin] PATH_INFO: `/v3/users/e50265cf1f244d62ac5e46117d6bce82` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:06.776609 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef5c9c4-cdd7-42cf-bb0d-94d3cb09a095 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e50265cf1f244d62ac5e46117d6bce82)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:06.807895 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef5c9c4-cdd7-42cf-bb0d-94d3cb09a095 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:06.812343 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 891/1778] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jul 15 19:36:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:36:06.815816 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8f59db-74c1-4a34-ac38-f1765756a95f admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:06.845568 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8f59db-74c1-4a34-ac38-f1765756a95f admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:06.859759 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa12b9aa-d25b-4f8e-b743-3fbcb577dc68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:06.860453 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa12b9aa-d25b-4f8e-b743-3fbcb577dc68 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:06.860725 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa12b9aa-d25b-4f8e-b743-3fbcb577dc68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:06.861072 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa12b9aa-d25b-4f8e-b743-3fbcb577dc68 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:06.861935 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa12b9aa-d25b-4f8e-b743-3fbcb577dc68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:06.866676 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa12b9aa-d25b-4f8e-b743-3fbcb577dc68 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:07.017065 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3ef5c9c4-cdd7-42cf-bb0d-94d3cb09a095 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e50265cf1f244d62ac5e46117d6bce82'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:07.022513 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 892/1779] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jul 15 19:36:06 2022] POST /identity/v3/projects => generated 363 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:36:07.024322 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa12b9aa-d25b-4f8e-b743-3fbcb577dc68 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:07.060177 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3ef5c9c4-cdd7-42cf-bb0d-94d3cb09a095 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e50265cf1f244d62ac5e46117d6bce82'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:07.167646 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3ef5c9c4-cdd7-42cf-bb0d-94d3cb09a095 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e50265cf1f244d62ac5e46117d6bce82'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:07.170388 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:36:07.171711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3ef5c9c4-cdd7-42cf-bb0d-94d3cb09a095 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e50265cf1f244d62ac5e46117d6bce82'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:07.182924 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3ef5c9c4-cdd7-42cf-bb0d-94d3cb09a095 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e50265cf1f244d62ac5e46117d6bce82'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:07.211615 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 888/1780] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jul 15 19:36:06 2022] DELETE /identity/v3/users/e50265cf1f244d62ac5e46117d6bce82 => generated 0 bytes in 582 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:36:07.216734 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0768a9c9-92fa-4a3c-bfc3-ab132fdf664f None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:07.217079 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0768a9c9-92fa-4a3c-bfc3-ab132fdf664f None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:07.217378 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0768a9c9-92fa-4a3c-bfc3-ab132fdf664f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:07.241634 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa12b9aa-d25b-4f8e-b743-3fbcb577dc68 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:07.265514 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-0768a9c9-92fa-4a3c-bfc3-ab132fdf664f None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:36:07.265514 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0768a9c9-92fa-4a3c-bfc3-ab132fdf664f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:36:07.373046 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ddeea3-eb43-406e-9ebe-260cb1443113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:07.375280 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ddeea3-eb43-406e-9ebe-260cb1443113 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:07.375957 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ddeea3-eb43-406e-9ebe-260cb1443113 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:07.377378 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ddeea3-eb43-406e-9ebe-260cb1443113 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:07.378345 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ddeea3-eb43-406e-9ebe-260cb1443113 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:07.383080 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ddeea3-eb43-406e-9ebe-260cb1443113 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:07.428900 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 889/1781] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:36:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:36:07.492486 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0768a9c9-92fa-4a3c-bfc3-ab132fdf664f None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:07.494231 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0768a9c9-92fa-4a3c-bfc3-ab132fdf664f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:36:07.538813 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 893/1782] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jul 15 19:36:07 2022] POST /identity/v3/users => generated 282 bytes in 513 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:36:07.540357 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ddeea3-eb43-406e-9ebe-260cb1443113 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:07.542352 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-c9ddeea3-eb43-406e-9ebe-260cb1443113 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:36:07.557350 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ddeea3-eb43-406e-9ebe-260cb1443113 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:07.574704 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7438cf37-7a9c-4923-91a5-d5a973843dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:07.575449 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7438cf37-7a9c-4923-91a5-d5a973843dce admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:07.575741 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7438cf37-7a9c-4923-91a5-d5a973843dce admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:07.580957 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7438cf37-7a9c-4923-91a5-d5a973843dce admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:07.580957 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7438cf37-7a9c-4923-91a5-d5a973843dce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:07.580957 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7438cf37-7a9c-4923-91a5-d5a973843dce admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:07.598959 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 894/1783] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:36:07 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:36:07.600002 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7438cf37-7a9c-4923-91a5-d5a973843dce admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:07.619650 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7438cf37-7a9c-4923-91a5-d5a973843dce admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:07.634865 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcec4ba-9aff-4cf2-9721-d0069572bba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:07.635570 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcec4ba-9aff-4cf2-9721-d0069572bba8 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:07.635859 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcec4ba-9aff-4cf2-9721-d0069572bba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:07.636139 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcec4ba-9aff-4cf2-9721-d0069572bba8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:07.637253 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcec4ba-9aff-4cf2-9721-d0069572bba8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:07.638896 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcec4ba-9aff-4cf2-9721-d0069572bba8 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:07.650613 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0768a9c9-92fa-4a3c-bfc3-ab132fdf664f None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:07.654882 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 895/1784] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:36:07 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:36:07.659857 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcec4ba-9aff-4cf2-9721-d0069572bba8 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:07.666296 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56518ce-6187-424b-8274-63c9135817e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:07.666873 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56518ce-6187-424b-8274-63c9135817e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:07.672804 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56518ce-6187-424b-8274-63c9135817e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:07.672804 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56518ce-6187-424b-8274-63c9135817e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:07.682332 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcec4ba-9aff-4cf2-9721-d0069572bba8 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:07.685951 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56518ce-6187-424b-8274-63c9135817e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:07.687624 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56518ce-6187-424b-8274-63c9135817e7 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:07.695914 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402ea664-2772-49db-a73b-c529e77d1893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:07.696623 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402ea664-2772-49db-a73b-c529e77d1893 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:07.697499 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402ea664-2772-49db-a73b-c529e77d1893 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:07.698593 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402ea664-2772-49db-a73b-c529e77d1893 admin admin] PATH_INFO: `/v3/projects/41cf16a5996142068ee01b49b0109ce6/users/ff68dc423e0049e38d15ab09a4951e22/roles/28763c6e70094116b3a4b2895895fa58` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:07.701035 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 890/1785] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:36:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 19:36:07.745834 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402ea664-2772-49db-a73b-c529e77d1893 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41cf16a5996142068ee01b49b0109ce6, user_id=ff68dc423e0049e38d15ab09a4951e22, role_id=28763c6e70094116b3a4b2895895fa58)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:07.749196 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402ea664-2772-49db-a73b-c529e77d1893 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:07.871515 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 896/1786] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:36:07 2022] PUT /identity/v3/projects/41cf16a5996142068ee01b49b0109ce6/users/ff68dc423e0049e38d15ab09a4951e22/roles/28763c6e70094116b3a4b2895895fa58 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:36:07.883384 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d215a136-21b3-4a8b-8566-bbc7e8a1f45f None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:07.883963 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d215a136-21b3-4a8b-8566-bbc7e8a1f45f None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:07.884113 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d215a136-21b3-4a8b-8566-bbc7e8a1f45f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:07.990724 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-d215a136-21b3-4a8b-8566-bbc7e8a1f45f None None] MFA Rules not processed for user `ff68dc423e0049e38d15ab09a4951e22`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:36:08.004713 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d215a136-21b3-4a8b-8566-bbc7e8a1f45f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:36:08.103544 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:36:08.184621 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 891/1787] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:36:07 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:36:08.187696 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402ea664-2772-49db-a73b-c529e77d1893 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:08.412447 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402ea664-2772-49db-a73b-c529e77d1893 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:08.436086 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9902a818-e7b4-4086-9c7e-3c62f06dbbdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:08.437270 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9902a818-e7b4-4086-9c7e-3c62f06dbbdb admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:08.437710 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9902a818-e7b4-4086-9c7e-3c62f06dbbdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:08.438909 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9902a818-e7b4-4086-9c7e-3c62f06dbbdb admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:08.439794 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9902a818-e7b4-4086-9c7e-3c62f06dbbdb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:08.448993 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9902a818-e7b4-4086-9c7e-3c62f06dbbdb admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:08.606462 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 897/1788] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jul 15 19:36:08 2022] POST /identity/v3/projects => generated 365 bytes in 419 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:36:08.610569 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d215a136-21b3-4a8b-8566-bbc7e8a1f45f None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:08.627356 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d215a136-21b3-4a8b-8566-bbc7e8a1f45f None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:08.780284 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac103376-6f06-4681-98de-917c495ec326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:08.780637 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac103376-6f06-4681-98de-917c495ec326 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:08.781055 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac103376-6f06-4681-98de-917c495ec326 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:08.781593 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac103376-6f06-4681-98de-917c495ec326 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:08.782301 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac103376-6f06-4681-98de-917c495ec326 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:08.784466 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac103376-6f06-4681-98de-917c495ec326 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:08.892507 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 892/1789] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jul 15 19:36:08 2022] POST /identity/v3/users => generated 289 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:36:08.896654 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9902a818-e7b4-4086-9c7e-3c62f06dbbdb admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:08.912415 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9902a818-e7b4-4086-9c7e-3c62f06dbbdb admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:08.931341 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1598b03d-18d2-4c83-a1db-3515e2f74690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:08.932288 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1598b03d-18d2-4c83-a1db-3515e2f74690 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:08.932725 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1598b03d-18d2-4c83-a1db-3515e2f74690 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:08.934088 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1598b03d-18d2-4c83-a1db-3515e2f74690 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:08.934088 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1598b03d-18d2-4c83-a1db-3515e2f74690 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:08.938149 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1598b03d-18d2-4c83-a1db-3515e2f74690 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:08.955937 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 898/1790] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:36:08 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:36:08.960091 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac103376-6f06-4681-98de-917c495ec326 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:08.973102 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac103376-6f06-4681-98de-917c495ec326 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:09.000137 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192fcebf-a309-4974-97ee-1c6cfefbcbfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:09.001298 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192fcebf-a309-4974-97ee-1c6cfefbcbfc admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:09.001707 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192fcebf-a309-4974-97ee-1c6cfefbcbfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:09.002134 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192fcebf-a309-4974-97ee-1c6cfefbcbfc admin admin] PATH_INFO: `/v3/projects/b4264e570c89442eb82b49a07df3270a/users/1dad2fb1ee7b467fa440675b79cac32b/roles/4318f86ba9f74cd4a0bef1c311276657` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:09.038501 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192fcebf-a309-4974-97ee-1c6cfefbcbfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4264e570c89442eb82b49a07df3270a, user_id=1dad2fb1ee7b467fa440675b79cac32b, role_id=4318f86ba9f74cd4a0bef1c311276657)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:09.043797 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:36:09.045286 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192fcebf-a309-4974-97ee-1c6cfefbcbfc admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:09.149604 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 893/1791] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:36:08 2022] PUT /identity/v3/projects/b4264e570c89442eb82b49a07df3270a/users/1dad2fb1ee7b467fa440675b79cac32b/roles/4318f86ba9f74cd4a0bef1c311276657 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:36:09.152348 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1598b03d-18d2-4c83-a1db-3515e2f74690 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:09.178636 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1598b03d-18d2-4c83-a1db-3515e2f74690 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:09.361390 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822343df-598f-42af-9910-0b3ea81f1311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:09.362283 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822343df-598f-42af-9910-0b3ea81f1311 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:09.365129 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822343df-598f-42af-9910-0b3ea81f1311 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:09.365509 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822343df-598f-42af-9910-0b3ea81f1311 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:09.366356 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822343df-598f-42af-9910-0b3ea81f1311 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:09.368455 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822343df-598f-42af-9910-0b3ea81f1311 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:09.397134 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 899/1792] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:36:09 2022] GET /identity/v3/roles => generated 1420 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:36:09.398473 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192fcebf-a309-4974-97ee-1c6cfefbcbfc admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:09.418905 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192fcebf-a309-4974-97ee-1c6cfefbcbfc admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:09.429523 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfa4ad7-6af7-42b3-9b60-c9fdd1f52a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:09.431383 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfa4ad7-6af7-42b3-9b60-c9fdd1f52a49 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:09.432050 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfa4ad7-6af7-42b3-9b60-c9fdd1f52a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:09.432568 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfa4ad7-6af7-42b3-9b60-c9fdd1f52a49 admin admin] PATH_INFO: `/v3/projects/b4264e570c89442eb82b49a07df3270a/users/1dad2fb1ee7b467fa440675b79cac32b/roles/4318f86ba9f74cd4a0bef1c311276657` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:09.435883 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfa4ad7-6af7-42b3-9b60-c9fdd1f52a49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4264e570c89442eb82b49a07df3270a, user_id=1dad2fb1ee7b467fa440675b79cac32b, role_id=4318f86ba9f74cd4a0bef1c311276657)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:09.448441 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfa4ad7-6af7-42b3-9b60-c9fdd1f52a49 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:09.487096 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 894/1793] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:36:09 2022] PUT /identity/v3/projects/b4264e570c89442eb82b49a07df3270a/users/1dad2fb1ee7b467fa440675b79cac32b/roles/4318f86ba9f74cd4a0bef1c311276657 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:36:09.492614 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940bb292-a377-4361-b741-07b87241230d None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:09.493153 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940bb292-a377-4361-b741-07b87241230d None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:09.493973 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940bb292-a377-4361-b741-07b87241230d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:09.560605 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-940bb292-a377-4361-b741-07b87241230d None None] MFA Rules not processed for user `1dad2fb1ee7b467fa440675b79cac32b`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:36:09.563724 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-940bb292-a377-4361-b741-07b87241230d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:36:09.745740 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 900/1794] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:36:09 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:36:09.761284 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfa4ad7-6af7-42b3-9b60-c9fdd1f52a49 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:09.763992 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-edfa4ad7-6af7-42b3-9b60-c9fdd1f52a49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:36:09.883255 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfa4ad7-6af7-42b3-9b60-c9fdd1f52a49 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:09.889208 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45df878e-41d1-4952-ac29-24c9027fd65b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86c2df3d378490ca7ea497a73c19d40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:09.894442 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df878e-41d1-4952-ac29-24c9027fd65b service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:09.894442 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df878e-41d1-4952-ac29-24c9027fd65b service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:09.894442 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df878e-41d1-4952-ac29-24c9027fd65b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:09.903501 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940bb292-a377-4361-b741-07b87241230d None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:09.906436 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-940bb292-a377-4361-b741-07b87241230d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:36:09.933412 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45df878e-41d1-4952-ac29-24c9027fd65b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:09.935640 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45df878e-41d1-4952-ac29-24c9027fd65b service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:09.983151 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 895/1795] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:36:09 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 19:36:10.003290 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:36:10.075646 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940bb292-a377-4361-b741-07b87241230d None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:10.094293 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f48ae01-fc33-4687-8aeb-5165536b4134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:10.096876 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f48ae01-fc33-4687-8aeb-5165536b4134 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:10.097340 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f48ae01-fc33-4687-8aeb-5165536b4134 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:10.097627 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f48ae01-fc33-4687-8aeb-5165536b4134 admin admin] PATH_INFO: `/v3/projects/3b4769ea3e1445328fade580459a8ff4` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:10.099130 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f48ae01-fc33-4687-8aeb-5165536b4134 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b4769ea3e1445328fade580459a8ff4)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:10.102610 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f48ae01-fc33-4687-8aeb-5165536b4134 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:10.328208 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-9f48ae01-fc33-4687-8aeb-5165536b4134 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b4769ea3e1445328fade580459a8ff4', 'request_id': 'req-9f48ae01-fc33-4687-8aeb-5165536b4134'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:10.404051 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45df878e-41d1-4952-ac29-24c9027fd65b service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:10.407833 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45df878e-41d1-4952-ac29-24c9027fd65b service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:10.534989 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbad3c5-fdad-4f5b-9528-eda395408b5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86c2df3d378490ca7ea497a73c19d40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:10.541474 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbad3c5-fdad-4f5b-9528-eda395408b5d service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:10.541864 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbad3c5-fdad-4f5b-9528-eda395408b5d service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:10.542224 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbad3c5-fdad-4f5b-9528-eda395408b5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:10.583010 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-9f48ae01-fc33-4687-8aeb-5165536b4134 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b4769ea3e1445328fade580459a8ff4', 'request_id': 'req-9f48ae01-fc33-4687-8aeb-5165536b4134'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:10.595252 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-9f48ae01-fc33-4687-8aeb-5165536b4134 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b4769ea3e1445328fade580459a8ff4', 'request_id': 'req-9f48ae01-fc33-4687-8aeb-5165536b4134'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:10.621300 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-9f48ae01-fc33-4687-8aeb-5165536b4134 admin admin] The token cache is being invalidate because project 3b4769ea3e1445328fade580459a8ff4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:36:10.621712 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-9f48ae01-fc33-4687-8aeb-5165536b4134 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:10.622504 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-9f48ae01-fc33-4687-8aeb-5165536b4134 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:10.627370 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 901/1796] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jul 15 19:36:09 2022] DELETE /identity/v3/projects/3b4769ea3e1445328fade580459a8ff4 => generated 0 bytes in 721 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:36:10.628486 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f48ae01-fc33-4687-8aeb-5165536b4134 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:10.630800 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-9f48ae01-fc33-4687-8aeb-5165536b4134 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:36:10.691017 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbad3c5-fdad-4f5b-9528-eda395408b5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:10.694859 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbad3c5-fdad-4f5b-9528-eda395408b5d service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:10.729680 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 896/1797] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:36:10 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 19:36:10.729883 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f48ae01-fc33-4687-8aeb-5165536b4134 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:10.737053 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904f07bb-fa0b-4c55-ac72-abf5da6afd72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:10.737053 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f07bb-fa0b-4c55-ac72-abf5da6afd72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:10.737529 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f07bb-fa0b-4c55-ac72-abf5da6afd72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:10.737994 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f07bb-fa0b-4c55-ac72-abf5da6afd72 admin admin] PATH_INFO: `/v3/users/9001a5ff5fd84f35b1cf5a634b264e30` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:10.739488 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904f07bb-fa0b-4c55-ac72-abf5da6afd72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9001a5ff5fd84f35b1cf5a634b264e30)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:10.741990 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904f07bb-fa0b-4c55-ac72-abf5da6afd72 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:11.335401 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-904f07bb-fa0b-4c55-ac72-abf5da6afd72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9001a5ff5fd84f35b1cf5a634b264e30'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:11.349680 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-904f07bb-fa0b-4c55-ac72-abf5da6afd72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9001a5ff5fd84f35b1cf5a634b264e30'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:11.492731 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-904f07bb-fa0b-4c55-ac72-abf5da6afd72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9001a5ff5fd84f35b1cf5a634b264e30'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:11.492731 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-904f07bb-fa0b-4c55-ac72-abf5da6afd72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9001a5ff5fd84f35b1cf5a634b264e30'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:11.492731 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-904f07bb-fa0b-4c55-ac72-abf5da6afd72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9001a5ff5fd84f35b1cf5a634b264e30'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:11.505156 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 902/1798] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jul 15 19:36:10 2022] DELETE /identity/v3/users/9001a5ff5fd84f35b1cf5a634b264e30 => generated 0 bytes in 877 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:36:12.086178 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbad3c5-fdad-4f5b-9528-eda395408b5d service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:12.099141 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbad3c5-fdad-4f5b-9528-eda395408b5d service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:12.161723 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626380f1-de31-4f79-9de8-8f30bdf90439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:12.162679 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626380f1-de31-4f79-9de8-8f30bdf90439 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:12.162978 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626380f1-de31-4f79-9de8-8f30bdf90439 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:12.163255 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626380f1-de31-4f79-9de8-8f30bdf90439 admin admin] PATH_INFO: `/v3/projects/32c5433b12fd49cfb776e256cd7f6fa3` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:12.164494 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626380f1-de31-4f79-9de8-8f30bdf90439 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32c5433b12fd49cfb776e256cd7f6fa3)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:12.166513 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626380f1-de31-4f79-9de8-8f30bdf90439 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:12.319262 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:36:12.327945 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-626380f1-de31-4f79-9de8-8f30bdf90439 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32c5433b12fd49cfb776e256cd7f6fa3', 'request_id': 'req-626380f1-de31-4f79-9de8-8f30bdf90439'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:12.438787 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-626380f1-de31-4f79-9de8-8f30bdf90439 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32c5433b12fd49cfb776e256cd7f6fa3', 'request_id': 'req-626380f1-de31-4f79-9de8-8f30bdf90439'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:12.445352 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-626380f1-de31-4f79-9de8-8f30bdf90439 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32c5433b12fd49cfb776e256cd7f6fa3', 'request_id': 'req-626380f1-de31-4f79-9de8-8f30bdf90439'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:12.457160 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-626380f1-de31-4f79-9de8-8f30bdf90439 admin admin] The token cache is being invalidate because project 32c5433b12fd49cfb776e256cd7f6fa3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:36:12.458004 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-626380f1-de31-4f79-9de8-8f30bdf90439 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:12.458680 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-626380f1-de31-4f79-9de8-8f30bdf90439 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:12.460295 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 897/1799] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jul 15 19:36:12 2022] DELETE /identity/v3/projects/32c5433b12fd49cfb776e256cd7f6fa3 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:36:12.467103 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb54a8-cd03-4fec-b04d-2ed9ea808e87 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:12.467580 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb54a8-cd03-4fec-b04d-2ed9ea808e87 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:12.467683 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb54a8-cd03-4fec-b04d-2ed9ea808e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:12.485812 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-22fb54a8-cd03-4fec-b04d-2ed9ea808e87 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:36:12.487017 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-22fb54a8-cd03-4fec-b04d-2ed9ea808e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:36:12.547681 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 903/1800] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:36:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:36:12.556748 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b60a99-7f02-4737-92ff-e979a637cbd9 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:12.556959 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b60a99-7f02-4737-92ff-e979a637cbd9 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:12.557385 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b60a99-7f02-4737-92ff-e979a637cbd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:12.576994 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-44b60a99-7f02-4737-92ff-e979a637cbd9 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:36:12.578232 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-44b60a99-7f02-4737-92ff-e979a637cbd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:36:12.588183 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 898/1801] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:36:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:36:12.592567 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fb54a8-cd03-4fec-b04d-2ed9ea808e87 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:12.602719 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fb54a8-cd03-4fec-b04d-2ed9ea808e87 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:12.607459 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7f29e1-8a9d-4acb-aaf7-00944d5f9321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:12.608125 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7f29e1-8a9d-4acb-aaf7-00944d5f9321 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:12.608413 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7f29e1-8a9d-4acb-aaf7-00944d5f9321 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:12.608687 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7f29e1-8a9d-4acb-aaf7-00944d5f9321 admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:12.609481 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7f29e1-8a9d-4acb-aaf7-00944d5f9321 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:36:12.609809 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7f29e1-8a9d-4acb-aaf7-00944d5f9321 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:12.611408 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7f29e1-8a9d-4acb-aaf7-00944d5f9321 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:12.623678 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 904/1802] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jul 15 19:36:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:36:12.628633 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b60a99-7f02-4737-92ff-e979a637cbd9 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:12.638788 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b60a99-7f02-4737-92ff-e979a637cbd9 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:12.643620 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7864fd-5984-4cf6-b83b-4fee1994e6f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:12.644285 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7864fd-5984-4cf6-b83b-4fee1994e6f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:12.644562 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7864fd-5984-4cf6-b83b-4fee1994e6f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:12.645454 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7864fd-5984-4cf6-b83b-4fee1994e6f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:12.646180 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7864fd-5984-4cf6-b83b-4fee1994e6f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:12.648070 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7864fd-5984-4cf6-b83b-4fee1994e6f4 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:12.751843 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 899/1803] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jul 15 19:36:12 2022] POST /identity/v3/projects => generated 385 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:36:12.756685 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7f29e1-8a9d-4acb-aaf7-00944d5f9321 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:12.767085 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7f29e1-8a9d-4acb-aaf7-00944d5f9321 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:12.827173 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488fef8f-1341-49b1-a051-4f139bd7898f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:12.832234 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488fef8f-1341-49b1-a051-4f139bd7898f admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:12.832234 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488fef8f-1341-49b1-a051-4f139bd7898f admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:12.832234 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488fef8f-1341-49b1-a051-4f139bd7898f admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:12.832234 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488fef8f-1341-49b1-a051-4f139bd7898f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:12.832234 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488fef8f-1341-49b1-a051-4f139bd7898f admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:12.952048 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 905/1804] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jul 15 19:36:12 2022] POST /identity/v3/users => generated 293 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:36:12.953253 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7864fd-5984-4cf6-b83b-4fee1994e6f4 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:12.964232 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7864fd-5984-4cf6-b83b-4fee1994e6f4 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:12.969497 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fc9450-d721-43f9-b531-036e677be414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:12.970286 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fc9450-d721-43f9-b531-036e677be414 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:12.970495 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fc9450-d721-43f9-b531-036e677be414 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:12.970805 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fc9450-d721-43f9-b531-036e677be414 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:12.971539 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fc9450-d721-43f9-b531-036e677be414 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:12.973167 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fc9450-d721-43f9-b531-036e677be414 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:12.984464 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 900/1805] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:36:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:36:12.990752 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488fef8f-1341-49b1-a051-4f139bd7898f admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:13.001044 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488fef8f-1341-49b1-a051-4f139bd7898f admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:13.010005 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22194841-98ef-4636-8338-e56923250ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:13.010863 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22194841-98ef-4636-8338-e56923250ab4 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:13.011213 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22194841-98ef-4636-8338-e56923250ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:13.011802 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22194841-98ef-4636-8338-e56923250ab4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:13.012603 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22194841-98ef-4636-8338-e56923250ab4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:13.014521 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22194841-98ef-4636-8338-e56923250ab4 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:13.025358 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 906/1806] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:36:12 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 19:36:13.029724 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fc9450-d721-43f9-b531-036e677be414 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:13.041893 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fc9450-d721-43f9-b531-036e677be414 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:13.047643 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:36:13.049475 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e63b5a1-e659-44d0-9d33-129f80ad546e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:13.050438 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e63b5a1-e659-44d0-9d33-129f80ad546e admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:13.050725 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e63b5a1-e659-44d0-9d33-129f80ad546e admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:13.051575 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e63b5a1-e659-44d0-9d33-129f80ad546e admin admin] PATH_INFO: `/v3/projects/7a020ed9b2c441a5afbb81f66b8d29f5/users/114f09e10a55483d86c89c9df73fadd3/roles/28763c6e70094116b3a4b2895895fa58` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:13.072766 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e63b5a1-e659-44d0-9d33-129f80ad546e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a020ed9b2c441a5afbb81f66b8d29f5, user_id=114f09e10a55483d86c89c9df73fadd3, role_id=28763c6e70094116b3a4b2895895fa58)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:13.076454 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e63b5a1-e659-44d0-9d33-129f80ad546e admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:13.156685 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 901/1807] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:36:13 2022] PUT /identity/v3/projects/7a020ed9b2c441a5afbb81f66b8d29f5/users/114f09e10a55483d86c89c9df73fadd3/roles/28763c6e70094116b3a4b2895895fa58 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:36:13.170847 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d94d84-05ab-4a72-a065-43833986a780 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:13.171147 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d94d84-05ab-4a72-a065-43833986a780 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:13.171431 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d94d84-05ab-4a72-a065-43833986a780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:13.200276 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-21d94d84-05ab-4a72-a065-43833986a780 None None] MFA Rules not processed for user `114f09e10a55483d86c89c9df73fadd3`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:36:13.201679 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-21d94d84-05ab-4a72-a065-43833986a780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:36:13.273785 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 907/1808] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:36:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:36:13.284780 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e63b5a1-e659-44d0-9d33-129f80ad546e admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:13.287023 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9e63b5a1-e659-44d0-9d33-129f80ad546e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:36:13.348236 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e63b5a1-e659-44d0-9d33-129f80ad546e admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:13.352073 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6a2c44-b420-4bcc-ba64-567135d4ca98 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86c2df3d378490ca7ea497a73c19d40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:13.354801 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a2c44-b420-4bcc-ba64-567135d4ca98 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:13.354801 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a2c44-b420-4bcc-ba64-567135d4ca98 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:13.354801 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a2c44-b420-4bcc-ba64-567135d4ca98 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:13.363018 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6a2c44-b420-4bcc-ba64-567135d4ca98 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:13.365005 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6a2c44-b420-4bcc-ba64-567135d4ca98 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:13.374834 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 902/1809] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:36:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:36:14.037154 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:36:15.246436 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d94d84-05ab-4a72-a065-43833986a780 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:15.247844 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-21d94d84-05ab-4a72-a065-43833986a780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:36:15.315319 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d94d84-05ab-4a72-a065-43833986a780 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:15.320574 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923f7589-8e19-4836-9bfc-bebf9afb4467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:15.321510 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923f7589-8e19-4836-9bfc-bebf9afb4467 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:15.323200 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923f7589-8e19-4836-9bfc-bebf9afb4467 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:15.323200 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923f7589-8e19-4836-9bfc-bebf9afb4467 admin admin] PATH_INFO: `/v3/users/ff68dc423e0049e38d15ab09a4951e22` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:15.323434 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923f7589-8e19-4836-9bfc-bebf9afb4467 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff68dc423e0049e38d15ab09a4951e22)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:15.325280 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923f7589-8e19-4836-9bfc-bebf9afb4467 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:15.451281 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:36:15.544447 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-923f7589-8e19-4836-9bfc-bebf9afb4467 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff68dc423e0049e38d15ab09a4951e22'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:15.559596 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-923f7589-8e19-4836-9bfc-bebf9afb4467 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff68dc423e0049e38d15ab09a4951e22'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:15.642262 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-923f7589-8e19-4836-9bfc-bebf9afb4467 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff68dc423e0049e38d15ab09a4951e22'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:15.642953 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-923f7589-8e19-4836-9bfc-bebf9afb4467 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff68dc423e0049e38d15ab09a4951e22'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:15.643586 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-923f7589-8e19-4836-9bfc-bebf9afb4467 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff68dc423e0049e38d15ab09a4951e22'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:15.657030 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 908/1810] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jul 15 19:36:15 2022] DELETE /identity/v3/users/ff68dc423e0049e38d15ab09a4951e22 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:36:15.661724 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f61da6-d8b8-4c3b-9e4a-09856669e049 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:15.662043 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f61da6-d8b8-4c3b-9e4a-09856669e049 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:15.663816 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f61da6-d8b8-4c3b-9e4a-09856669e049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:15.682441 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-e6f61da6-d8b8-4c3b-9e4a-09856669e049 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:36:15.684586 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e6f61da6-d8b8-4c3b-9e4a-09856669e049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:36:15.756177 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 903/1811] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:36:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:36:15.766424 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923f7589-8e19-4836-9bfc-bebf9afb4467 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:15.767866 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-923f7589-8e19-4836-9bfc-bebf9afb4467 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:36:15.832225 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923f7589-8e19-4836-9bfc-bebf9afb4467 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:15.835931 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bfe14b-c724-4dd8-8ac2-40baee0012bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:15.836591 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bfe14b-c724-4dd8-8ac2-40baee0012bb service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:15.836872 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bfe14b-c724-4dd8-8ac2-40baee0012bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:15.837154 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bfe14b-c724-4dd8-8ac2-40baee0012bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:15.848781 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bfe14b-c724-4dd8-8ac2-40baee0012bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:15.851104 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bfe14b-c724-4dd8-8ac2-40baee0012bb service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:15.863961 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 909/1812] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:36:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:36:15.905522 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:36:17.541861 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f61da6-d8b8-4c3b-9e4a-09856669e049 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:17.543474 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e6f61da6-d8b8-4c3b-9e4a-09856669e049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:36:17.555715 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f61da6-d8b8-4c3b-9e4a-09856669e049 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:17.577943 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7020cd46-6867-48a5-a677-56713f3008b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:17.578939 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7020cd46-6867-48a5-a677-56713f3008b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:17.579381 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7020cd46-6867-48a5-a677-56713f3008b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:17.579665 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7020cd46-6867-48a5-a677-56713f3008b6 admin admin] PATH_INFO: `/v3/projects/41cf16a5996142068ee01b49b0109ce6` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:17.582334 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7020cd46-6867-48a5-a677-56713f3008b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41cf16a5996142068ee01b49b0109ce6)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:17.585975 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7020cd46-6867-48a5-a677-56713f3008b6 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:17.720385 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-7020cd46-6867-48a5-a677-56713f3008b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41cf16a5996142068ee01b49b0109ce6', 'request_id': 'req-7020cd46-6867-48a5-a677-56713f3008b6'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:17.815794 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-7020cd46-6867-48a5-a677-56713f3008b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41cf16a5996142068ee01b49b0109ce6', 'request_id': 'req-7020cd46-6867-48a5-a677-56713f3008b6'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:17.824740 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-7020cd46-6867-48a5-a677-56713f3008b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41cf16a5996142068ee01b49b0109ce6', 'request_id': 'req-7020cd46-6867-48a5-a677-56713f3008b6'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:17.844369 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-7020cd46-6867-48a5-a677-56713f3008b6 admin admin] The token cache is being invalidate because project 41cf16a5996142068ee01b49b0109ce6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:36:17.849217 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-7020cd46-6867-48a5-a677-56713f3008b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:17.849217 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-7020cd46-6867-48a5-a677-56713f3008b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:17.849217 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 904/1813] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jul 15 19:36:17 2022] DELETE /identity/v3/projects/41cf16a5996142068ee01b49b0109ce6 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:36:17.851838 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bfe14b-c724-4dd8-8ac2-40baee0012bb service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:17.861547 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bfe14b-c724-4dd8-8ac2-40baee0012bb service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:17.918937 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b1fd02-45d5-4632-b903-bc9ddfafc04b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:17.920090 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b1fd02-45d5-4632-b903-bc9ddfafc04b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:17.920369 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b1fd02-45d5-4632-b903-bc9ddfafc04b admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:17.920641 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b1fd02-45d5-4632-b903-bc9ddfafc04b admin admin] PATH_INFO: `/v3/users/1dad2fb1ee7b467fa440675b79cac32b` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:17.921850 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b1fd02-45d5-4632-b903-bc9ddfafc04b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1dad2fb1ee7b467fa440675b79cac32b)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:17.924426 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b1fd02-45d5-4632-b903-bc9ddfafc04b admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:18.132083 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-d2b1fd02-45d5-4632-b903-bc9ddfafc04b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1dad2fb1ee7b467fa440675b79cac32b'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:18.146242 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-d2b1fd02-45d5-4632-b903-bc9ddfafc04b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1dad2fb1ee7b467fa440675b79cac32b'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:18.238293 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-d2b1fd02-45d5-4632-b903-bc9ddfafc04b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1dad2fb1ee7b467fa440675b79cac32b'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:18.238949 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-d2b1fd02-45d5-4632-b903-bc9ddfafc04b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1dad2fb1ee7b467fa440675b79cac32b'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:18.240205 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-d2b1fd02-45d5-4632-b903-bc9ddfafc04b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1dad2fb1ee7b467fa440675b79cac32b'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:18.253176 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 910/1814] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jul 15 19:36:17 2022] DELETE /identity/v3/users/1dad2fb1ee7b467fa440675b79cac32b => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:36:18.302593 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:36:18.562603 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:36:18.682239 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7020cd46-6867-48a5-a677-56713f3008b6 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:18.682239 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-7020cd46-6867-48a5-a677-56713f3008b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:36:18.693127 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7020cd46-6867-48a5-a677-56713f3008b6 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:18.759254 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e9d95e-966a-48ee-bdfd-5dcc37e25201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:18.768835 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e9d95e-966a-48ee-bdfd-5dcc37e25201 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:18.768835 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e9d95e-966a-48ee-bdfd-5dcc37e25201 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:18.768835 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e9d95e-966a-48ee-bdfd-5dcc37e25201 admin admin] PATH_INFO: `/v3/projects/b4264e570c89442eb82b49a07df3270a` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:18.768835 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e9d95e-966a-48ee-bdfd-5dcc37e25201 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4264e570c89442eb82b49a07df3270a)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:18.768835 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e9d95e-966a-48ee-bdfd-5dcc37e25201 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:18.868230 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-c3e9d95e-966a-48ee-bdfd-5dcc37e25201 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4264e570c89442eb82b49a07df3270a', 'request_id': 'req-c3e9d95e-966a-48ee-bdfd-5dcc37e25201'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:18.941605 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-c3e9d95e-966a-48ee-bdfd-5dcc37e25201 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4264e570c89442eb82b49a07df3270a', 'request_id': 'req-c3e9d95e-966a-48ee-bdfd-5dcc37e25201'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:18.949045 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-c3e9d95e-966a-48ee-bdfd-5dcc37e25201 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4264e570c89442eb82b49a07df3270a', 'request_id': 'req-c3e9d95e-966a-48ee-bdfd-5dcc37e25201'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:18.962186 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-c3e9d95e-966a-48ee-bdfd-5dcc37e25201 admin admin] The token cache is being invalidate because project b4264e570c89442eb82b49a07df3270a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:36:18.962477 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-c3e9d95e-966a-48ee-bdfd-5dcc37e25201 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:18.963426 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-c3e9d95e-966a-48ee-bdfd-5dcc37e25201 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:18.965085 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 905/1815] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jul 15 19:36:18 2022] DELETE /identity/v3/projects/b4264e570c89442eb82b49a07df3270a => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:36:18.970270 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a5f34a-5f09-41db-97a6-8b7942c852f9 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:18.977284 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a5f34a-5f09-41db-97a6-8b7942c852f9 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:18.977284 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a5f34a-5f09-41db-97a6-8b7942c852f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:19.010033 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-17a5f34a-5f09-41db-97a6-8b7942c852f9 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:36:19.010033 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-17a5f34a-5f09-41db-97a6-8b7942c852f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:36:19.071721 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 911/1816] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:36:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:36:19.079599 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47fa026-13e1-4a8d-a12b-e77411423784 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:19.079899 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47fa026-13e1-4a8d-a12b-e77411423784 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:19.080181 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47fa026-13e1-4a8d-a12b-e77411423784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:19.099218 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-d47fa026-13e1-4a8d-a12b-e77411423784 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:36:19.100368 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d47fa026-13e1-4a8d-a12b-e77411423784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:36:19.108517 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 906/1817] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:36:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:36:19.112757 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a5f34a-5f09-41db-97a6-8b7942c852f9 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:19.123289 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a5f34a-5f09-41db-97a6-8b7942c852f9 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:19.128243 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be452f1e-71d3-4e86-87f3-9ce98a7f3f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:19.132182 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be452f1e-71d3-4e86-87f3-9ce98a7f3f78 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:19.132182 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be452f1e-71d3-4e86-87f3-9ce98a7f3f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:19.132182 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be452f1e-71d3-4e86-87f3-9ce98a7f3f78 admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:19.132182 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be452f1e-71d3-4e86-87f3-9ce98a7f3f78 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:36:19.132182 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be452f1e-71d3-4e86-87f3-9ce98a7f3f78 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:19.132822 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be452f1e-71d3-4e86-87f3-9ce98a7f3f78 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:19.146923 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 912/1818] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jul 15 19:36:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:36:19.151712 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47fa026-13e1-4a8d-a12b-e77411423784 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:19.161059 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47fa026-13e1-4a8d-a12b-e77411423784 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:19.165856 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f681a8eb-b15d-4c63-8544-57588971b11c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:19.189239 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f681a8eb-b15d-4c63-8544-57588971b11c admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:19.189239 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f681a8eb-b15d-4c63-8544-57588971b11c admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:19.189239 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f681a8eb-b15d-4c63-8544-57588971b11c admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:19.189239 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f681a8eb-b15d-4c63-8544-57588971b11c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:19.189239 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f681a8eb-b15d-4c63-8544-57588971b11c admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:19.297900 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:36:19.313426 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 907/1819] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jul 15 19:36:19 2022] POST /identity/v3/projects => generated 385 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:36:19.314504 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be452f1e-71d3-4e86-87f3-9ce98a7f3f78 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:19.326283 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be452f1e-71d3-4e86-87f3-9ce98a7f3f78 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:19.385029 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7ebf7f-5c4d-4b2a-9cad-da432cbf952f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:19.385671 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7ebf7f-5c4d-4b2a-9cad-da432cbf952f admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:19.386019 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7ebf7f-5c4d-4b2a-9cad-da432cbf952f admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:19.386567 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7ebf7f-5c4d-4b2a-9cad-da432cbf952f admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:19.387267 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7ebf7f-5c4d-4b2a-9cad-da432cbf952f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:19.389216 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7ebf7f-5c4d-4b2a-9cad-da432cbf952f admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:19.511361 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 913/1820] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jul 15 19:36:19 2022] POST /identity/v3/users => generated 293 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:36:19.515511 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f681a8eb-b15d-4c63-8544-57588971b11c admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:19.526142 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f681a8eb-b15d-4c63-8544-57588971b11c admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:19.531389 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8a7b72-f90c-489c-977d-22ae0fbffbc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:19.532089 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8a7b72-f90c-489c-977d-22ae0fbffbc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:19.532373 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8a7b72-f90c-489c-977d-22ae0fbffbc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:19.532647 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8a7b72-f90c-489c-977d-22ae0fbffbc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:19.533382 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8a7b72-f90c-489c-977d-22ae0fbffbc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:19.535143 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8a7b72-f90c-489c-977d-22ae0fbffbc9 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:19.546726 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 908/1821] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:36:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:36:19.550728 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7ebf7f-5c4d-4b2a-9cad-da432cbf952f admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:19.560699 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7ebf7f-5c4d-4b2a-9cad-da432cbf952f admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:19.565592 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4937f5-b92e-4249-bb4e-0b9c818248de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:19.572749 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4937f5-b92e-4249-bb4e-0b9c818248de admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:19.572749 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4937f5-b92e-4249-bb4e-0b9c818248de admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:19.572749 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4937f5-b92e-4249-bb4e-0b9c818248de admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:19.572749 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4937f5-b92e-4249-bb4e-0b9c818248de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:19.572749 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4937f5-b92e-4249-bb4e-0b9c818248de admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:19.579864 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 914/1822] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:36:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:36:19.584528 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8a7b72-f90c-489c-977d-22ae0fbffbc9 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:19.594655 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8a7b72-f90c-489c-977d-22ae0fbffbc9 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:19.600228 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1376eefb-da24-470e-9c21-62c5357ef27f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:19.601120 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1376eefb-da24-470e-9c21-62c5357ef27f admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:19.606911 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1376eefb-da24-470e-9c21-62c5357ef27f admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:19.606911 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1376eefb-da24-470e-9c21-62c5357ef27f admin admin] PATH_INFO: `/v3/projects/fb2f923a934e480cb8fd8040d02ea678/users/88d4ec23fc81412c907de0315a7f83f6/roles/28763c6e70094116b3a4b2895895fa58` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:19.616752 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1376eefb-da24-470e-9c21-62c5357ef27f admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb2f923a934e480cb8fd8040d02ea678, user_id=88d4ec23fc81412c907de0315a7f83f6, role_id=28763c6e70094116b3a4b2895895fa58)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:19.619453 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1376eefb-da24-470e-9c21-62c5357ef27f admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:19.694556 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 909/1823] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:36:19 2022] PUT /identity/v3/projects/fb2f923a934e480cb8fd8040d02ea678/users/88d4ec23fc81412c907de0315a7f83f6/roles/28763c6e70094116b3a4b2895895fa58 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:36:19.705981 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e944bf61-8800-46ac-9f46-4eed7c7c7e1a None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:19.706278 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e944bf61-8800-46ac-9f46-4eed7c7c7e1a None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:19.706562 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e944bf61-8800-46ac-9f46-4eed7c7c7e1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:19.737430 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-e944bf61-8800-46ac-9f46-4eed7c7c7e1a None None] MFA Rules not processed for user `88d4ec23fc81412c907de0315a7f83f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:36:19.738949 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-e944bf61-8800-46ac-9f46-4eed7c7c7e1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:36:19.825815 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 915/1824] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:36:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:36:19.827309 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1376eefb-da24-470e-9c21-62c5357ef27f admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:19.894630 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1376eefb-da24-470e-9c21-62c5357ef27f admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:19.900366 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c208916b-1927-4595-ac86-a983948e9ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:19.901035 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c208916b-1927-4595-ac86-a983948e9ceb admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:19.901321 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c208916b-1927-4595-ac86-a983948e9ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:19.901604 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c208916b-1927-4595-ac86-a983948e9ceb admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:19.902357 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c208916b-1927-4595-ac86-a983948e9ceb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:19.904452 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c208916b-1927-4595-ac86-a983948e9ceb admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:20.011434 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e944bf61-8800-46ac-9f46-4eed7c7c7e1a None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:20.011943 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 910/1825] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jul 15 19:36:19 2022] POST /identity/v3/projects => generated 387 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:36:20.025322 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e944bf61-8800-46ac-9f46-4eed7c7c7e1a None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:20.095148 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2982c9a-56cd-47c0-8c19-6a93d96b2f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:20.095924 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2982c9a-56cd-47c0-8c19-6a93d96b2f99 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:20.096278 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2982c9a-56cd-47c0-8c19-6a93d96b2f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:20.096619 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2982c9a-56cd-47c0-8c19-6a93d96b2f99 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:20.097449 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2982c9a-56cd-47c0-8c19-6a93d96b2f99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:20.099967 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2982c9a-56cd-47c0-8c19-6a93d96b2f99 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:20.183743 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 916/1826] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jul 15 19:36:20 2022] POST /identity/v3/users => generated 300 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:36:20.190970 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c208916b-1927-4595-ac86-a983948e9ceb admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:20.205331 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c208916b-1927-4595-ac86-a983948e9ceb admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:20.211045 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e949886-6b04-4ca2-93f4-80b932bca2e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:20.212270 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e949886-6b04-4ca2-93f4-80b932bca2e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:20.212270 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e949886-6b04-4ca2-93f4-80b932bca2e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:20.212270 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e949886-6b04-4ca2-93f4-80b932bca2e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:20.212991 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e949886-6b04-4ca2-93f4-80b932bca2e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:20.214670 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e949886-6b04-4ca2-93f4-80b932bca2e3 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:20.228018 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 911/1827] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:36:20 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:36:20.231862 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2982c9a-56cd-47c0-8c19-6a93d96b2f99 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:20.240378 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:36:20.245199 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2982c9a-56cd-47c0-8c19-6a93d96b2f99 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:20.250381 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1d7659-bc8f-4597-9638-c2a7c872605c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:20.251461 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1d7659-bc8f-4597-9638-c2a7c872605c admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:20.251772 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1d7659-bc8f-4597-9638-c2a7c872605c admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:20.252147 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1d7659-bc8f-4597-9638-c2a7c872605c admin admin] PATH_INFO: `/v3/projects/203b8568beac4513939a2de5b571430f/users/65de83b3095a450a8be8c6194d709b0b/roles/4318f86ba9f74cd4a0bef1c311276657` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:20.271118 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1d7659-bc8f-4597-9638-c2a7c872605c admin admin] RBAC: Authorizing `identity:create_grant(project_id=203b8568beac4513939a2de5b571430f, user_id=65de83b3095a450a8be8c6194d709b0b, role_id=4318f86ba9f74cd4a0bef1c311276657)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:20.274006 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1d7659-bc8f-4597-9638-c2a7c872605c admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:20.307532 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:INSERT=3,keystone:DELETE=3,keystone:SELECT=94 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:36:20.362947 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 917/1828] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:36:20 2022] PUT /identity/v3/projects/203b8568beac4513939a2de5b571430f/users/65de83b3095a450a8be8c6194d709b0b/roles/4318f86ba9f74cd4a0bef1c311276657 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:36:20.367320 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e949886-6b04-4ca2-93f4-80b932bca2e3 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:20.377423 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e949886-6b04-4ca2-93f4-80b932bca2e3 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:20.437900 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e75f95-967e-4643-8750-28f70de12ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:20.438583 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e75f95-967e-4643-8750-28f70de12ab3 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:20.438871 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e75f95-967e-4643-8750-28f70de12ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:20.439150 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e75f95-967e-4643-8750-28f70de12ab3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:20.439897 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e75f95-967e-4643-8750-28f70de12ab3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:20.441482 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e75f95-967e-4643-8750-28f70de12ab3 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:20.454018 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 912/1829] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:36:20 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:36:20.468467 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1d7659-bc8f-4597-9638-c2a7c872605c admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:20.479872 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1d7659-bc8f-4597-9638-c2a7c872605c admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:20.484829 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0fa647-bb99-41c0-a5d9-3d08888fc5cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:20.488517 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0fa647-bb99-41c0-a5d9-3d08888fc5cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:20.488517 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0fa647-bb99-41c0-a5d9-3d08888fc5cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:20.488517 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0fa647-bb99-41c0-a5d9-3d08888fc5cd admin admin] PATH_INFO: `/v3/projects/203b8568beac4513939a2de5b571430f/users/65de83b3095a450a8be8c6194d709b0b/roles/4318f86ba9f74cd4a0bef1c311276657` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:20.489068 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0fa647-bb99-41c0-a5d9-3d08888fc5cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=203b8568beac4513939a2de5b571430f, user_id=65de83b3095a450a8be8c6194d709b0b, role_id=4318f86ba9f74cd4a0bef1c311276657)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:20.491837 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0fa647-bb99-41c0-a5d9-3d08888fc5cd admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:20.502797 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 918/1830] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:36:20 2022] PUT /identity/v3/projects/203b8568beac4513939a2de5b571430f/users/65de83b3095a450a8be8c6194d709b0b/roles/4318f86ba9f74cd4a0bef1c311276657 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:36:20.513559 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0bb14e-f4db-42f2-9071-f49d98daa62a None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:20.513922 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0bb14e-f4db-42f2-9071-f49d98daa62a None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:20.514202 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0bb14e-f4db-42f2-9071-f49d98daa62a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:20.543743 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-3b0bb14e-f4db-42f2-9071-f49d98daa62a None None] MFA Rules not processed for user `65de83b3095a450a8be8c6194d709b0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:36:20.545208 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-3b0bb14e-f4db-42f2-9071-f49d98daa62a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:36:20.618128 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 913/1831] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:36:20 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:36:20.628786 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0fa647-bb99-41c0-a5d9-3d08888fc5cd admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:20.630671 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-3f0fa647-bb99-41c0-a5d9-3d08888fc5cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:36:20.691137 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0fa647-bb99-41c0-a5d9-3d08888fc5cd admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:20.695326 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74c11ac-673a-4f17-bafe-0862386e3a49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86c2df3d378490ca7ea497a73c19d40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:20.695998 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74c11ac-673a-4f17-bafe-0862386e3a49 service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:20.696300 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74c11ac-673a-4f17-bafe-0862386e3a49 service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:20.696660 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74c11ac-673a-4f17-bafe-0862386e3a49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:20.706526 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74c11ac-673a-4f17-bafe-0862386e3a49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:20.707983 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74c11ac-673a-4f17-bafe-0862386e3a49 service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:20.717685 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 919/1832] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:36:20 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:36:21.004316 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:36:21.042752 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:36:21.425150 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:36:21.825089 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0bb14e-f4db-42f2-9071-f49d98daa62a None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:21.889794 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0bb14e-f4db-42f2-9071-f49d98daa62a None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:21.893977 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a138494-17a0-4784-bfd6-992d30918211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:21.894870 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a138494-17a0-4784-bfd6-992d30918211 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:21.895153 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a138494-17a0-4784-bfd6-992d30918211 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:21.895488 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a138494-17a0-4784-bfd6-992d30918211 admin admin] PATH_INFO: `/v3/users/88d4ec23fc81412c907de0315a7f83f6` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:21.896695 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a138494-17a0-4784-bfd6-992d30918211 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88d4ec23fc81412c907de0315a7f83f6)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:21.898684 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a138494-17a0-4784-bfd6-992d30918211 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:22.099192 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-2a138494-17a0-4784-bfd6-992d30918211 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88d4ec23fc81412c907de0315a7f83f6'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:22.112156 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-2a138494-17a0-4784-bfd6-992d30918211 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88d4ec23fc81412c907de0315a7f83f6'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:22.227235 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-2a138494-17a0-4784-bfd6-992d30918211 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88d4ec23fc81412c907de0315a7f83f6'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:22.227789 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-2a138494-17a0-4784-bfd6-992d30918211 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88d4ec23fc81412c907de0315a7f83f6'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:22.235459 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-2a138494-17a0-4784-bfd6-992d30918211 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88d4ec23fc81412c907de0315a7f83f6'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:22.242260 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 914/1833] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jul 15 19:36:21 2022] DELETE /identity/v3/users/88d4ec23fc81412c907de0315a7f83f6 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:36:22.246786 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4381d42d-0c21-4ad0-80cd-2fbba1f62805 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:22.247270 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4381d42d-0c21-4ad0-80cd-2fbba1f62805 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:22.247364 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4381d42d-0c21-4ad0-80cd-2fbba1f62805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:22.265967 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-4381d42d-0c21-4ad0-80cd-2fbba1f62805 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:36:22.267164 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-4381d42d-0c21-4ad0-80cd-2fbba1f62805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:36:22.271147 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:36:22.324964 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 920/1834] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:36:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:36:22.334662 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a138494-17a0-4784-bfd6-992d30918211 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:22.336057 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-2a138494-17a0-4784-bfd6-992d30918211 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:36:22.382658 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:36:22.395294 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a138494-17a0-4784-bfd6-992d30918211 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:22.399705 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0469155e-3098-4407-8160-85f1262816ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:22.400361 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0469155e-3098-4407-8160-85f1262816ac service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:22.400640 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0469155e-3098-4407-8160-85f1262816ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:22.400914 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0469155e-3098-4407-8160-85f1262816ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:22.411146 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0469155e-3098-4407-8160-85f1262816ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:22.413321 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0469155e-3098-4407-8160-85f1262816ac service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:22.423925 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 915/1835] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:36:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:36:22.960956 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4381d42d-0c21-4ad0-80cd-2fbba1f62805 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:22.963591 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-4381d42d-0c21-4ad0-80cd-2fbba1f62805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:36:22.982137 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4381d42d-0c21-4ad0-80cd-2fbba1f62805 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:22.987310 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c597d1c-5779-48d7-8273-d733c6837580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:22.988199 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c597d1c-5779-48d7-8273-d733c6837580 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:22.988487 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c597d1c-5779-48d7-8273-d733c6837580 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:22.988770 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c597d1c-5779-48d7-8273-d733c6837580 admin admin] PATH_INFO: `/v3/projects/fb2f923a934e480cb8fd8040d02ea678` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:22.990082 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c597d1c-5779-48d7-8273-d733c6837580 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb2f923a934e480cb8fd8040d02ea678)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:22.992153 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c597d1c-5779-48d7-8273-d733c6837580 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:23.119091 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-1c597d1c-5779-48d7-8273-d733c6837580 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb2f923a934e480cb8fd8040d02ea678', 'request_id': 'req-1c597d1c-5779-48d7-8273-d733c6837580'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:23.217440 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-1c597d1c-5779-48d7-8273-d733c6837580 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb2f923a934e480cb8fd8040d02ea678', 'request_id': 'req-1c597d1c-5779-48d7-8273-d733c6837580'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:23.222524 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-1c597d1c-5779-48d7-8273-d733c6837580 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb2f923a934e480cb8fd8040d02ea678', 'request_id': 'req-1c597d1c-5779-48d7-8273-d733c6837580'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:23.234359 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-1c597d1c-5779-48d7-8273-d733c6837580 admin admin] The token cache is being invalidate because project fb2f923a934e480cb8fd8040d02ea678 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:36:23.234648 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-1c597d1c-5779-48d7-8273-d733c6837580 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:23.235445 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-1c597d1c-5779-48d7-8273-d733c6837580 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:23.237236 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 921/1836] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jul 15 19:36:22 2022] DELETE /identity/v3/projects/fb2f923a934e480cb8fd8040d02ea678 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:36:23.242495 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0469155e-3098-4407-8160-85f1262816ac service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:23.254024 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0469155e-3098-4407-8160-85f1262816ac service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:23.315414 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8342dc-9c5a-4fda-b418-2c8febe79837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:23.316324 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8342dc-9c5a-4fda-b418-2c8febe79837 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:23.316606 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8342dc-9c5a-4fda-b418-2c8febe79837 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:23.316876 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8342dc-9c5a-4fda-b418-2c8febe79837 admin admin] PATH_INFO: `/v3/users/65de83b3095a450a8be8c6194d709b0b` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:23.318703 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8342dc-9c5a-4fda-b418-2c8febe79837 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65de83b3095a450a8be8c6194d709b0b)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:23.320640 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8342dc-9c5a-4fda-b418-2c8febe79837 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:23.619441 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-8f8342dc-9c5a-4fda-b418-2c8febe79837 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65de83b3095a450a8be8c6194d709b0b'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:23.633095 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-8f8342dc-9c5a-4fda-b418-2c8febe79837 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65de83b3095a450a8be8c6194d709b0b'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:23.684081 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-8f8342dc-9c5a-4fda-b418-2c8febe79837 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65de83b3095a450a8be8c6194d709b0b'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:23.684776 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-8f8342dc-9c5a-4fda-b418-2c8febe79837 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65de83b3095a450a8be8c6194d709b0b'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:23.693434 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-8f8342dc-9c5a-4fda-b418-2c8febe79837 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65de83b3095a450a8be8c6194d709b0b'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:23.699867 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 916/1837] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jul 15 19:36:23 2022] DELETE /identity/v3/users/65de83b3095a450a8be8c6194d709b0b => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:36:24.199276 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c597d1c-5779-48d7-8273-d733c6837580 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:24.200848 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-1c597d1c-5779-48d7-8273-d733c6837580 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:36:24.217730 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c597d1c-5779-48d7-8273-d733c6837580 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:24.279807 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9033aae3-9cb1-4ea6-8d49-db3347fae098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:24.280681 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9033aae3-9cb1-4ea6-8d49-db3347fae098 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:24.280963 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9033aae3-9cb1-4ea6-8d49-db3347fae098 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:24.281239 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9033aae3-9cb1-4ea6-8d49-db3347fae098 admin admin] PATH_INFO: `/v3/projects/203b8568beac4513939a2de5b571430f` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:24.282443 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9033aae3-9cb1-4ea6-8d49-db3347fae098 admin admin] RBAC: Authorizing `identity:delete_project(project_id=203b8568beac4513939a2de5b571430f)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:24.284376 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9033aae3-9cb1-4ea6-8d49-db3347fae098 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:24.397379 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:36:24.406853 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-9033aae3-9cb1-4ea6-8d49-db3347fae098 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '203b8568beac4513939a2de5b571430f', 'request_id': 'req-9033aae3-9cb1-4ea6-8d49-db3347fae098'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:24.524400 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-9033aae3-9cb1-4ea6-8d49-db3347fae098 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '203b8568beac4513939a2de5b571430f', 'request_id': 'req-9033aae3-9cb1-4ea6-8d49-db3347fae098'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:24.532380 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-9033aae3-9cb1-4ea6-8d49-db3347fae098 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '203b8568beac4513939a2de5b571430f', 'request_id': 'req-9033aae3-9cb1-4ea6-8d49-db3347fae098'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:24.543875 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-9033aae3-9cb1-4ea6-8d49-db3347fae098 admin admin] The token cache is being invalidate because project 203b8568beac4513939a2de5b571430f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:36:24.544161 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-9033aae3-9cb1-4ea6-8d49-db3347fae098 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:24.544811 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-9033aae3-9cb1-4ea6-8d49-db3347fae098 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:24.546408 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 922/1838] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jul 15 19:36:24 2022] DELETE /identity/v3/projects/203b8568beac4513939a2de5b571430f => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:36:24.551462 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92ff016-0402-43ff-a4cd-3ee03d266d67 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:24.552548 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92ff016-0402-43ff-a4cd-3ee03d266d67 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:24.552686 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92ff016-0402-43ff-a4cd-3ee03d266d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:24.570277 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-c92ff016-0402-43ff-a4cd-3ee03d266d67 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:36:24.571514 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-c92ff016-0402-43ff-a4cd-3ee03d266d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:36:24.631266 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 917/1839] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:36:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:36:24.639407 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c43dd80-6e10-41f7-bd12-dbef61b9aab8 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:24.639699 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c43dd80-6e10-41f7-bd12-dbef61b9aab8 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:24.639979 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c43dd80-6e10-41f7-bd12-dbef61b9aab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:24.658509 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-4c43dd80-6e10-41f7-bd12-dbef61b9aab8 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:36:24.659648 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-4c43dd80-6e10-41f7-bd12-dbef61b9aab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:36:24.668141 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 923/1840] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:36:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:36:24.672481 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92ff016-0402-43ff-a4cd-3ee03d266d67 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:24.678577 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:36:24.682250 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92ff016-0402-43ff-a4cd-3ee03d266d67 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:24.686890 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2209fa-a4fb-4b40-95ec-77f42acde023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:24.710237 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2209fa-a4fb-4b40-95ec-77f42acde023 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:24.710237 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2209fa-a4fb-4b40-95ec-77f42acde023 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:24.710237 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2209fa-a4fb-4b40-95ec-77f42acde023 admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:24.710237 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2209fa-a4fb-4b40-95ec-77f42acde023 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:36:24.710237 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2209fa-a4fb-4b40-95ec-77f42acde023 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:24.710237 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2209fa-a4fb-4b40-95ec-77f42acde023 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:24.710708 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 918/1841] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jul 15 19:36:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:36:24.713258 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c43dd80-6e10-41f7-bd12-dbef61b9aab8 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:24.722837 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c43dd80-6e10-41f7-bd12-dbef61b9aab8 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:24.728278 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c73157-9a89-4980-98dd-f49a9f519142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:24.728933 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c73157-9a89-4980-98dd-f49a9f519142 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:24.729217 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c73157-9a89-4980-98dd-f49a9f519142 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:24.729491 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c73157-9a89-4980-98dd-f49a9f519142 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:24.730463 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c73157-9a89-4980-98dd-f49a9f519142 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:24.732364 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c73157-9a89-4980-98dd-f49a9f519142 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:24.847591 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 924/1842] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jul 15 19:36:24 2022] POST /identity/v3/projects => generated 383 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:36:24.850504 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2209fa-a4fb-4b40-95ec-77f42acde023 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:24.860745 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2209fa-a4fb-4b40-95ec-77f42acde023 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:24.928804 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c90ef4-af13-450b-9218-6d02d83b3637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:24.929489 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c90ef4-af13-450b-9218-6d02d83b3637 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:24.929787 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c90ef4-af13-450b-9218-6d02d83b3637 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:24.930080 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c90ef4-af13-450b-9218-6d02d83b3637 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:24.930789 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c90ef4-af13-450b-9218-6d02d83b3637 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:24.932846 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c90ef4-af13-450b-9218-6d02d83b3637 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:25.073790 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 919/1843] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jul 15 19:36:24 2022] POST /identity/v3/users => generated 292 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:36:25.077939 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c73157-9a89-4980-98dd-f49a9f519142 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:25.090009 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c73157-9a89-4980-98dd-f49a9f519142 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:25.093718 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009b98c4-4a3f-4f22-b5b0-35adbe555513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:25.094507 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009b98c4-4a3f-4f22-b5b0-35adbe555513 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:25.094793 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009b98c4-4a3f-4f22-b5b0-35adbe555513 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:25.095072 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009b98c4-4a3f-4f22-b5b0-35adbe555513 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:25.095804 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009b98c4-4a3f-4f22-b5b0-35adbe555513 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:25.097406 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009b98c4-4a3f-4f22-b5b0-35adbe555513 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:25.108164 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 925/1844] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:36:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:36:25.111908 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c90ef4-af13-450b-9218-6d02d83b3637 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:25.122140 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c90ef4-af13-450b-9218-6d02d83b3637 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:25.127100 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c21055-081f-42a6-8f65-14c731761872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:25.127828 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c21055-081f-42a6-8f65-14c731761872 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:25.128123 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c21055-081f-42a6-8f65-14c731761872 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:25.128413 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c21055-081f-42a6-8f65-14c731761872 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:25.129200 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c21055-081f-42a6-8f65-14c731761872 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:25.131328 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c21055-081f-42a6-8f65-14c731761872 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:25.142360 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 920/1845] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:36:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:36:25.147301 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009b98c4-4a3f-4f22-b5b0-35adbe555513 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:25.157547 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009b98c4-4a3f-4f22-b5b0-35adbe555513 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:25.162171 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12d2453-6948-4a03-b4a8-f0538ba02912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:25.163062 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d2453-6948-4a03-b4a8-f0538ba02912 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:25.163337 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d2453-6948-4a03-b4a8-f0538ba02912 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:25.163609 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d2453-6948-4a03-b4a8-f0538ba02912 admin admin] PATH_INFO: `/v3/projects/52b7f5b1548749708474ce058ee4091a/users/8672e699bb1d41a68932b39ba176b404/roles/28763c6e70094116b3a4b2895895fa58` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:25.179400 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12d2453-6948-4a03-b4a8-f0538ba02912 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52b7f5b1548749708474ce058ee4091a, user_id=8672e699bb1d41a68932b39ba176b404, role_id=28763c6e70094116b3a4b2895895fa58)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:25.182062 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12d2453-6948-4a03-b4a8-f0538ba02912 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:25.249831 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 926/1846] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:36:25 2022] PUT /identity/v3/projects/52b7f5b1548749708474ce058ee4091a/users/8672e699bb1d41a68932b39ba176b404/roles/28763c6e70094116b3a4b2895895fa58 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:36:25.263750 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b770a948-906d-420c-8aae-8e6b7d88b813 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:25.264045 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b770a948-906d-420c-8aae-8e6b7d88b813 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:25.264323 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b770a948-906d-420c-8aae-8e6b7d88b813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:25.291821 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-b770a948-906d-420c-8aae-8e6b7d88b813 None None] MFA Rules not processed for user `8672e699bb1d41a68932b39ba176b404`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:36:25.293015 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b770a948-906d-420c-8aae-8e6b7d88b813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:36:25.355588 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 921/1847] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:36:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:36:25.360158 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12d2453-6948-4a03-b4a8-f0538ba02912 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:25.421189 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12d2453-6948-4a03-b4a8-f0538ba02912 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:25.425803 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8905375c-38da-4ca6-91db-158fde32e3af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:25.426455 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8905375c-38da-4ca6-91db-158fde32e3af admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:25.427265 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8905375c-38da-4ca6-91db-158fde32e3af admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:25.427535 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8905375c-38da-4ca6-91db-158fde32e3af admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:25.428226 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8905375c-38da-4ca6-91db-158fde32e3af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:25.430255 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8905375c-38da-4ca6-91db-158fde32e3af admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:25.543294 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 927/1848] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jul 15 19:36:25 2022] POST /identity/v3/projects => generated 385 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:36:25.547222 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b770a948-906d-420c-8aae-8e6b7d88b813 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:25.559196 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b770a948-906d-420c-8aae-8e6b7d88b813 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:25.617257 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cff1e15-c5ef-4b84-985a-d0c7c9087614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:25.617974 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cff1e15-c5ef-4b84-985a-d0c7c9087614 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:25.618257 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cff1e15-c5ef-4b84-985a-d0c7c9087614 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:25.618529 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cff1e15-c5ef-4b84-985a-d0c7c9087614 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:25.619223 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cff1e15-c5ef-4b84-985a-d0c7c9087614 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:25.621187 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cff1e15-c5ef-4b84-985a-d0c7c9087614 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:25.628152 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:36:25.705891 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 922/1849] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jul 15 19:36:25 2022] POST /identity/v3/users => generated 299 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:36:25.709803 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8905375c-38da-4ca6-91db-158fde32e3af admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:25.720738 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8905375c-38da-4ca6-91db-158fde32e3af admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:25.726453 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0e97e7-f01e-48f7-b750-5f4f521cae24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:25.727149 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0e97e7-f01e-48f7-b750-5f4f521cae24 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:25.731855 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0e97e7-f01e-48f7-b750-5f4f521cae24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:25.731855 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0e97e7-f01e-48f7-b750-5f4f521cae24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:25.731855 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0e97e7-f01e-48f7-b750-5f4f521cae24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:25.731855 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0e97e7-f01e-48f7-b750-5f4f521cae24 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:25.742757 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 928/1850] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:36:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:36:25.747019 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cff1e15-c5ef-4b84-985a-d0c7c9087614 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:25.757353 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cff1e15-c5ef-4b84-985a-d0c7c9087614 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:25.763450 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d950f0f-2299-4666-a8df-a59b3876896b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:25.764376 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d950f0f-2299-4666-a8df-a59b3876896b admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:25.764913 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d950f0f-2299-4666-a8df-a59b3876896b admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:25.765223 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d950f0f-2299-4666-a8df-a59b3876896b admin admin] PATH_INFO: `/v3/projects/cfcac46621424b08a8fb141a5d5bf736/users/c32ae423848f42049d07ee43c4fd2c1e/roles/4318f86ba9f74cd4a0bef1c311276657` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:25.782931 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d950f0f-2299-4666-a8df-a59b3876896b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfcac46621424b08a8fb141a5d5bf736, user_id=c32ae423848f42049d07ee43c4fd2c1e, role_id=4318f86ba9f74cd4a0bef1c311276657)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:25.786526 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d950f0f-2299-4666-a8df-a59b3876896b admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:25.876935 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 923/1851] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:36:25 2022] PUT /identity/v3/projects/cfcac46621424b08a8fb141a5d5bf736/users/c32ae423848f42049d07ee43c4fd2c1e/roles/4318f86ba9f74cd4a0bef1c311276657 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:36:25.880853 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0e97e7-f01e-48f7-b750-5f4f521cae24 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:25.897989 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0e97e7-f01e-48f7-b750-5f4f521cae24 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:25.929328 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:36:25.955190 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583dfc28-2301-4022-95b6-16d590f62df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:25.955884 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583dfc28-2301-4022-95b6-16d590f62df9 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:25.956251 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583dfc28-2301-4022-95b6-16d590f62df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:25.956523 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583dfc28-2301-4022-95b6-16d590f62df9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:25.957176 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583dfc28-2301-4022-95b6-16d590f62df9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:25.958867 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583dfc28-2301-4022-95b6-16d590f62df9 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:25.970046 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 929/1852] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:36:25 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:36:25.974620 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d950f0f-2299-4666-a8df-a59b3876896b admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:25.985270 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d950f0f-2299-4666-a8df-a59b3876896b admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:25.990760 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d017e9-a69f-4aa8-8574-74343b47f549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:25.991644 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d017e9-a69f-4aa8-8574-74343b47f549 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:25.991927 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d017e9-a69f-4aa8-8574-74343b47f549 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:25.992201 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d017e9-a69f-4aa8-8574-74343b47f549 admin admin] PATH_INFO: `/v3/projects/cfcac46621424b08a8fb141a5d5bf736/users/c32ae423848f42049d07ee43c4fd2c1e/roles/4318f86ba9f74cd4a0bef1c311276657` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:25.994228 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d017e9-a69f-4aa8-8574-74343b47f549 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfcac46621424b08a8fb141a5d5bf736, user_id=c32ae423848f42049d07ee43c4fd2c1e, role_id=4318f86ba9f74cd4a0bef1c311276657)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:25.996943 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d017e9-a69f-4aa8-8574-74343b47f549 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:26.008167 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 924/1853] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:36:25 2022] PUT /identity/v3/projects/cfcac46621424b08a8fb141a5d5bf736/users/c32ae423848f42049d07ee43c4fd2c1e/roles/4318f86ba9f74cd4a0bef1c311276657 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:36:26.018850 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a932b3a4-817d-4d95-8857-c67872923b57 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:26.019141 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a932b3a4-817d-4d95-8857-c67872923b57 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:26.019421 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a932b3a4-817d-4d95-8857-c67872923b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:26.048278 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-a932b3a4-817d-4d95-8857-c67872923b57 None None] MFA Rules not processed for user `c32ae423848f42049d07ee43c4fd2c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:36:26.049599 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-a932b3a4-817d-4d95-8857-c67872923b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:36:26.132104 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 930/1854] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:36:26 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:36:26.143903 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d017e9-a69f-4aa8-8574-74343b47f549 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:26.146317 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e2d017e9-a69f-4aa8-8574-74343b47f549 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:36:26.206052 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d017e9-a69f-4aa8-8574-74343b47f549 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:26.210329 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9501874c-328b-4162-9cfd-c77628a9acf0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86c2df3d378490ca7ea497a73c19d40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:26.210921 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9501874c-328b-4162-9cfd-c77628a9acf0 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:26.211203 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9501874c-328b-4162-9cfd-c77628a9acf0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:26.211480 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9501874c-328b-4162-9cfd-c77628a9acf0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:26.282878 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9501874c-328b-4162-9cfd-c77628a9acf0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:26.284330 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9501874c-328b-4162-9cfd-c77628a9acf0 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:26.294504 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 925/1855] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:36:26 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:36:33.027605 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc82ad7c-7948-46df-8ea5-21ec2a91999f None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:33.028069 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc82ad7c-7948-46df-8ea5-21ec2a91999f None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:33.028170 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc82ad7c-7948-46df-8ea5-21ec2a91999f None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:33.029853 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 931/1856] 10.222.0.33 () {54 vars in 921 bytes} [Fri Jul 15 19:36:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:36:33.053116 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a77e9f-7d26-4982-837a-31bddde54dcf None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:33.053593 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a77e9f-7d26-4982-837a-31bddde54dcf None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:33.053716 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a77e9f-7d26-4982-837a-31bddde54dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:33.072625 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-85a77e9f-7d26-4982-837a-31bddde54dcf None None] MFA Rules not processed for user `7673a3b5c0324e07a9535ac70ed7a491`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:36:33.073972 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-85a77e9f-7d26-4982-837a-31bddde54dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:36:33.130634 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:36:33.146885 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 926/1857] 10.222.0.33 () {58 vars in 1018 bytes} [Fri Jul 15 19:36:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:36:33.285699 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc82ad7c-7948-46df-8ea5-21ec2a91999f None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:33.287576 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-bc82ad7c-7948-46df-8ea5-21ec2a91999f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:36:33.306000 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc82ad7c-7948-46df-8ea5-21ec2a91999f None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:33.311728 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8374bf29-9665-483d-b8e3-d938c58676db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7673a3b5c0324e07a9535ac70ed7a491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:33.312379 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8374bf29-9665-483d-b8e3-d938c58676db service nova] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:33.312658 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8374bf29-9665-483d-b8e3-d938c58676db service nova] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:33.312932 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8374bf29-9665-483d-b8e3-d938c58676db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:33.322651 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8374bf29-9665-483d-b8e3-d938c58676db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:33.324221 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8374bf29-9665-483d-b8e3-d938c58676db service nova] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:33.335297 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 932/1858] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:36:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:36:35.180514 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a77e9f-7d26-4982-837a-31bddde54dcf None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:35.182284 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-85a77e9f-7d26-4982-837a-31bddde54dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:36:35.273801 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a77e9f-7d26-4982-837a-31bddde54dcf None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:35.278602 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb9fd4f-060d-41ce-9c87-7bfcc5f9d3aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:35.279497 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb9fd4f-060d-41ce-9c87-7bfcc5f9d3aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:35.279781 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb9fd4f-060d-41ce-9c87-7bfcc5f9d3aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:35.280056 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb9fd4f-060d-41ce-9c87-7bfcc5f9d3aa admin admin] PATH_INFO: `/v3/users/114f09e10a55483d86c89c9df73fadd3` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:35.281437 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb9fd4f-060d-41ce-9c87-7bfcc5f9d3aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=114f09e10a55483d86c89c9df73fadd3)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:35.283445 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb9fd4f-060d-41ce-9c87-7bfcc5f9d3aa admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:35.637267 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6cb9fd4f-060d-41ce-9c87-7bfcc5f9d3aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '114f09e10a55483d86c89c9df73fadd3'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:35.651099 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6cb9fd4f-060d-41ce-9c87-7bfcc5f9d3aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '114f09e10a55483d86c89c9df73fadd3'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:35.717927 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6cb9fd4f-060d-41ce-9c87-7bfcc5f9d3aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '114f09e10a55483d86c89c9df73fadd3'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:35.718802 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6cb9fd4f-060d-41ce-9c87-7bfcc5f9d3aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '114f09e10a55483d86c89c9df73fadd3'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:35.730633 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6cb9fd4f-060d-41ce-9c87-7bfcc5f9d3aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '114f09e10a55483d86c89c9df73fadd3'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:35.738370 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 927/1859] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jul 15 19:36:35 2022] DELETE /identity/v3/users/114f09e10a55483d86c89c9df73fadd3 => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:36:35.743556 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf537bb0-965a-4e7b-a71d-2f32e3bd36f8 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:35.743996 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf537bb0-965a-4e7b-a71d-2f32e3bd36f8 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:35.744147 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf537bb0-965a-4e7b-a71d-2f32e3bd36f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:35.763560 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-cf537bb0-965a-4e7b-a71d-2f32e3bd36f8 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:36:35.765462 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-cf537bb0-965a-4e7b-a71d-2f32e3bd36f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:36:35.835886 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 933/1860] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:36:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:36:35.842724 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb9fd4f-060d-41ce-9c87-7bfcc5f9d3aa admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:35.844197 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-6cb9fd4f-060d-41ce-9c87-7bfcc5f9d3aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:36:35.902671 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb9fd4f-060d-41ce-9c87-7bfcc5f9d3aa admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:35.906298 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57144cdf-c102-41c0-8b30-3b944be3f86c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:35.906944 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57144cdf-c102-41c0-8b30-3b944be3f86c service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:35.907232 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57144cdf-c102-41c0-8b30-3b944be3f86c service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:35.907528 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57144cdf-c102-41c0-8b30-3b944be3f86c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:35.917256 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57144cdf-c102-41c0-8b30-3b944be3f86c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:35.918712 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57144cdf-c102-41c0-8b30-3b944be3f86c service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:35.928266 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 928/1861] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:36:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:36:36.472176 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf537bb0-965a-4e7b-a71d-2f32e3bd36f8 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:36.474365 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-cf537bb0-965a-4e7b-a71d-2f32e3bd36f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:36:36.486732 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf537bb0-965a-4e7b-a71d-2f32e3bd36f8 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:36.492476 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abc8a4c-462c-4969-8ce8-0cadaa1ed22d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:36.493448 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abc8a4c-462c-4969-8ce8-0cadaa1ed22d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:36.497727 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abc8a4c-462c-4969-8ce8-0cadaa1ed22d admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:36.497727 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abc8a4c-462c-4969-8ce8-0cadaa1ed22d admin admin] PATH_INFO: `/v3/projects/7a020ed9b2c441a5afbb81f66b8d29f5` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:36.497727 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abc8a4c-462c-4969-8ce8-0cadaa1ed22d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a020ed9b2c441a5afbb81f66b8d29f5)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:36.497727 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abc8a4c-462c-4969-8ce8-0cadaa1ed22d admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:36.626033 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:36:36.634813 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-4abc8a4c-462c-4969-8ce8-0cadaa1ed22d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a020ed9b2c441a5afbb81f66b8d29f5', 'request_id': 'req-4abc8a4c-462c-4969-8ce8-0cadaa1ed22d'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:36.740952 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-4abc8a4c-462c-4969-8ce8-0cadaa1ed22d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a020ed9b2c441a5afbb81f66b8d29f5', 'request_id': 'req-4abc8a4c-462c-4969-8ce8-0cadaa1ed22d'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:36.748257 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-4abc8a4c-462c-4969-8ce8-0cadaa1ed22d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a020ed9b2c441a5afbb81f66b8d29f5', 'request_id': 'req-4abc8a4c-462c-4969-8ce8-0cadaa1ed22d'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:36.759123 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-4abc8a4c-462c-4969-8ce8-0cadaa1ed22d admin admin] The token cache is being invalidate because project 7a020ed9b2c441a5afbb81f66b8d29f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:36:36.759414 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-4abc8a4c-462c-4969-8ce8-0cadaa1ed22d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:36.760080 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-4abc8a4c-462c-4969-8ce8-0cadaa1ed22d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:36.762021 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 934/1862] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jul 15 19:36:36 2022] DELETE /identity/v3/projects/7a020ed9b2c441a5afbb81f66b8d29f5 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:36:36.766902 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e072811a-3b85-49b7-8e58-5edc221474d4 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:36.767376 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e072811a-3b85-49b7-8e58-5edc221474d4 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:36.767452 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e072811a-3b85-49b7-8e58-5edc221474d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:36.788552 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-e072811a-3b85-49b7-8e58-5edc221474d4 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:36:36.789933 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e072811a-3b85-49b7-8e58-5edc221474d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:36:36.820126 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:36:36.867324 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 929/1863] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:36:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:36:36.875873 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f8e743-34b3-4c7b-9e20-af05c1312847 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:36.876164 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f8e743-34b3-4c7b-9e20-af05c1312847 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:36.876470 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f8e743-34b3-4c7b-9e20-af05c1312847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:36.895997 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-25f8e743-34b3-4c7b-9e20-af05c1312847 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:36:36.897192 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-25f8e743-34b3-4c7b-9e20-af05c1312847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:36:36.905625 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 935/1864] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:36:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:36:36.910143 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e072811a-3b85-49b7-8e58-5edc221474d4 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:36.920626 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e072811a-3b85-49b7-8e58-5edc221474d4 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:36.925584 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd16eafd-4af8-46f6-8bb2-066ebf9e142b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:36.926318 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd16eafd-4af8-46f6-8bb2-066ebf9e142b admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:36.926596 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd16eafd-4af8-46f6-8bb2-066ebf9e142b admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:36.926871 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd16eafd-4af8-46f6-8bb2-066ebf9e142b admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:36.927589 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd16eafd-4af8-46f6-8bb2-066ebf9e142b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:36:36.927889 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd16eafd-4af8-46f6-8bb2-066ebf9e142b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:36.929478 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd16eafd-4af8-46f6-8bb2-066ebf9e142b admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:36.948139 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 930/1865] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jul 15 19:36:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:36:36.953477 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f8e743-34b3-4c7b-9e20-af05c1312847 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:36.964388 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f8e743-34b3-4c7b-9e20-af05c1312847 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:36.969475 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b49ac25-a02b-48f8-b72c-0c74d1a8a0b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:36.970441 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b49ac25-a02b-48f8-b72c-0c74d1a8a0b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:36.971015 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b49ac25-a02b-48f8-b72c-0c74d1a8a0b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:36.971498 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b49ac25-a02b-48f8-b72c-0c74d1a8a0b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:36.972446 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b49ac25-a02b-48f8-b72c-0c74d1a8a0b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:36.975549 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b49ac25-a02b-48f8-b72c-0c74d1a8a0b9 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:37.086579 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 936/1866] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jul 15 19:36:36 2022] POST /identity/v3/projects => generated 375 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:36:37.090724 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd16eafd-4af8-46f6-8bb2-066ebf9e142b admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:37.100945 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd16eafd-4af8-46f6-8bb2-066ebf9e142b admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:37.156928 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8324d0d0-e83b-4dcf-9ba8-50198b945d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:37.157592 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8324d0d0-e83b-4dcf-9ba8-50198b945d1c admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:37.157902 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8324d0d0-e83b-4dcf-9ba8-50198b945d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:37.158178 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8324d0d0-e83b-4dcf-9ba8-50198b945d1c admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:37.158856 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8324d0d0-e83b-4dcf-9ba8-50198b945d1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:37.160965 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8324d0d0-e83b-4dcf-9ba8-50198b945d1c admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:37.328514 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 931/1867] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jul 15 19:36:37 2022] POST /identity/v3/users => generated 288 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:36:37.333080 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b49ac25-a02b-48f8-b72c-0c74d1a8a0b9 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:37.347035 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b49ac25-a02b-48f8-b72c-0c74d1a8a0b9 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:37.362555 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea8883c-c6ad-4f51-ac0e-2972aa6d5d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:37.363223 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea8883c-c6ad-4f51-ac0e-2972aa6d5d73 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:37.363507 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea8883c-c6ad-4f51-ac0e-2972aa6d5d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:37.364403 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea8883c-c6ad-4f51-ac0e-2972aa6d5d73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:37.368319 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea8883c-c6ad-4f51-ac0e-2972aa6d5d73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:37.368715 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8324d0d0-e83b-4dcf-9ba8-50198b945d1c admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:37.370204 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea8883c-c6ad-4f51-ac0e-2972aa6d5d73 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:37.370636 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-8324d0d0-e83b-4dcf-9ba8-50198b945d1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:36:37.385203 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 937/1868] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:36:37 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:36:37.386644 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea8883c-c6ad-4f51-ac0e-2972aa6d5d73 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:37.390071 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8324d0d0-e83b-4dcf-9ba8-50198b945d1c admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:37.399749 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea8883c-c6ad-4f51-ac0e-2972aa6d5d73 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:37.404018 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b763ef48-78b2-40c0-846b-426890bf7a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:37.406713 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b763ef48-78b2-40c0-846b-426890bf7a02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:37.406713 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b763ef48-78b2-40c0-846b-426890bf7a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:37.406713 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b763ef48-78b2-40c0-846b-426890bf7a02 admin admin] PATH_INFO: `/v3/users/dca75d34e96d49d5bff2b15c4fb26fae` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:37.407887 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b763ef48-78b2-40c0-846b-426890bf7a02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dca75d34e96d49d5bff2b15c4fb26fae)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:37.412239 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b763ef48-78b2-40c0-846b-426890bf7a02 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:37.415636 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013a4463-c17e-43ee-a6ca-5b061a7e698c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:37.416450 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a4463-c17e-43ee-a6ca-5b061a7e698c admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:37.416932 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a4463-c17e-43ee-a6ca-5b061a7e698c admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:37.417282 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a4463-c17e-43ee-a6ca-5b061a7e698c admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:37.427407 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013a4463-c17e-43ee-a6ca-5b061a7e698c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:37.429025 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013a4463-c17e-43ee-a6ca-5b061a7e698c admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:37.445201 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 938/1869] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:36:37 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:36:37.446938 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013a4463-c17e-43ee-a6ca-5b061a7e698c admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:37.467605 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013a4463-c17e-43ee-a6ca-5b061a7e698c admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:37.480113 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737e49a9-80aa-4741-bb37-d402db8b9368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:37.481032 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737e49a9-80aa-4741-bb37-d402db8b9368 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:37.481345 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737e49a9-80aa-4741-bb37-d402db8b9368 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:37.481900 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737e49a9-80aa-4741-bb37-d402db8b9368 admin admin] PATH_INFO: `/v3/projects/92d449cf5d5d4c10a2d50bfe740312cb/users/e7ac8a635f044826b308b0b4d26e65c2/roles/28763c6e70094116b3a4b2895895fa58` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:37.538621 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737e49a9-80aa-4741-bb37-d402db8b9368 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92d449cf5d5d4c10a2d50bfe740312cb, user_id=e7ac8a635f044826b308b0b4d26e65c2, role_id=28763c6e70094116b3a4b2895895fa58)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:37.538621 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737e49a9-80aa-4741-bb37-d402db8b9368 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:37.639599 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 939/1870] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:36:37 2022] PUT /identity/v3/projects/92d449cf5d5d4c10a2d50bfe740312cb/users/e7ac8a635f044826b308b0b4d26e65c2/roles/28763c6e70094116b3a4b2895895fa58 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:36:37.650130 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39825981-559f-43eb-bcfa-ddb7e8b7ea1a None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:37.650421 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39825981-559f-43eb-bcfa-ddb7e8b7ea1a None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:37.650699 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39825981-559f-43eb-bcfa-ddb7e8b7ea1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:37.689960 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-39825981-559f-43eb-bcfa-ddb7e8b7ea1a None None] MFA Rules not processed for user `e7ac8a635f044826b308b0b4d26e65c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:36:37.691177 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-39825981-559f-43eb-bcfa-ddb7e8b7ea1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:36:37.716887 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b763ef48-78b2-40c0-846b-426890bf7a02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dca75d34e96d49d5bff2b15c4fb26fae'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:37.742451 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b763ef48-78b2-40c0-846b-426890bf7a02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dca75d34e96d49d5bff2b15c4fb26fae'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:37.782467 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 940/1871] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:36:37 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:36:37.823030 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b763ef48-78b2-40c0-846b-426890bf7a02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dca75d34e96d49d5bff2b15c4fb26fae'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:37.823801 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b763ef48-78b2-40c0-846b-426890bf7a02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dca75d34e96d49d5bff2b15c4fb26fae'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:37.832472 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b763ef48-78b2-40c0-846b-426890bf7a02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dca75d34e96d49d5bff2b15c4fb26fae'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:37.839324 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 932/1872] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jul 15 19:36:37 2022] DELETE /identity/v3/users/dca75d34e96d49d5bff2b15c4fb26fae => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:36:37.844897 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7656151d-330b-475f-8462-9c31a5591458 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:37.845202 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7656151d-330b-475f-8462-9c31a5591458 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:37.848379 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7656151d-330b-475f-8462-9c31a5591458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:37.863727 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-7656151d-330b-475f-8462-9c31a5591458 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:36:37.864987 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-7656151d-330b-475f-8462-9c31a5591458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:36:37.902130 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:36:37.914006 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b763ef48-78b2-40c0-846b-426890bf7a02 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:37.915553 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b763ef48-78b2-40c0-846b-426890bf7a02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:36:37.930534 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:36:37.938404 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 941/1873] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:36:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:36:37.951490 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7656151d-330b-475f-8462-9c31a5591458 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:37.955467 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-7656151d-330b-475f-8462-9c31a5591458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:36:37.986657 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b763ef48-78b2-40c0-846b-426890bf7a02 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:37.995138 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cbd572-1f95-436e-b9a5-a838de588538 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86c2df3d378490ca7ea497a73c19d40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:37.997126 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cbd572-1f95-436e-b9a5-a838de588538 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:37.997126 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cbd572-1f95-436e-b9a5-a838de588538 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:37.997126 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cbd572-1f95-436e-b9a5-a838de588538 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:37.998587 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d8cbd572-1f95-436e-b9a5-a838de588538 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:36:38.033916 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7656151d-330b-475f-8462-9c31a5591458 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:38.040389 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab27713d-41e3-4e13-ac44-e3db1f8bd262 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:38.041060 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab27713d-41e3-4e13-ac44-e3db1f8bd262 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:38.041353 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab27713d-41e3-4e13-ac44-e3db1f8bd262 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:38.041817 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab27713d-41e3-4e13-ac44-e3db1f8bd262 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:38.054911 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab27713d-41e3-4e13-ac44-e3db1f8bd262 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:38.056954 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab27713d-41e3-4e13-ac44-e3db1f8bd262 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:38.069135 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 942/1874] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:36:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 19:36:38.093863 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cbd572-1f95-436e-b9a5-a838de588538 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:38.095827 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cbd572-1f95-436e-b9a5-a838de588538 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:38.125977 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 933/1875] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:36:37 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 19:36:38.588555 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab27713d-41e3-4e13-ac44-e3db1f8bd262 service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:38.590183 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-ab27713d-41e3-4e13-ac44-e3db1f8bd262 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:36:38.605335 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab27713d-41e3-4e13-ac44-e3db1f8bd262 service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:38.615104 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ee7c89-62e1-4caa-aa69-e8ae54fee5d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:38.615714 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ee7c89-62e1-4caa-aa69-e8ae54fee5d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:38.620206 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ee7c89-62e1-4caa-aa69-e8ae54fee5d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:38.620206 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ee7c89-62e1-4caa-aa69-e8ae54fee5d7 admin admin] PATH_INFO: `/v3/projects/0d6b04e5b04f4b00be9b74a05d74ee04` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:38.620206 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ee7c89-62e1-4caa-aa69-e8ae54fee5d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d6b04e5b04f4b00be9b74a05d74ee04)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:38.620206 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ee7c89-62e1-4caa-aa69-e8ae54fee5d7 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:38.786139 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-f5ee7c89-62e1-4caa-aa69-e8ae54fee5d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d6b04e5b04f4b00be9b74a05d74ee04', 'request_id': 'req-f5ee7c89-62e1-4caa-aa69-e8ae54fee5d7'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:38.858889 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-f5ee7c89-62e1-4caa-aa69-e8ae54fee5d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d6b04e5b04f4b00be9b74a05d74ee04', 'request_id': 'req-f5ee7c89-62e1-4caa-aa69-e8ae54fee5d7'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:38.868209 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-f5ee7c89-62e1-4caa-aa69-e8ae54fee5d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d6b04e5b04f4b00be9b74a05d74ee04', 'request_id': 'req-f5ee7c89-62e1-4caa-aa69-e8ae54fee5d7'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:38.884157 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-f5ee7c89-62e1-4caa-aa69-e8ae54fee5d7 admin admin] The token cache is being invalidate because project 0d6b04e5b04f4b00be9b74a05d74ee04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:36:38.891798 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-f5ee7c89-62e1-4caa-aa69-e8ae54fee5d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:38.891798 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-f5ee7c89-62e1-4caa-aa69-e8ae54fee5d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:38.891798 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 943/1876] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jul 15 19:36:38 2022] DELETE /identity/v3/projects/0d6b04e5b04f4b00be9b74a05d74ee04 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:36:38.894339 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cbd572-1f95-436e-b9a5-a838de588538 service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:38.907436 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cbd572-1f95-436e-b9a5-a838de588538 service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:38.974675 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20780ba-580b-4494-a269-57d042e2ea34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:38.975785 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20780ba-580b-4494-a269-57d042e2ea34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:38.976195 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20780ba-580b-4494-a269-57d042e2ea34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:38.978159 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20780ba-580b-4494-a269-57d042e2ea34 admin admin] PATH_INFO: `/v3/users/611152a8389a436884553fbaba6b7447` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:38.978159 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20780ba-580b-4494-a269-57d042e2ea34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=611152a8389a436884553fbaba6b7447)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:38.979970 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20780ba-580b-4494-a269-57d042e2ea34 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:39.235654 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-a20780ba-580b-4494-a269-57d042e2ea34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '611152a8389a436884553fbaba6b7447'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:39.249456 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-a20780ba-580b-4494-a269-57d042e2ea34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '611152a8389a436884553fbaba6b7447'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:39.251717 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:36:39.321393 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-a20780ba-580b-4494-a269-57d042e2ea34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '611152a8389a436884553fbaba6b7447'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:39.322062 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-a20780ba-580b-4494-a269-57d042e2ea34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '611152a8389a436884553fbaba6b7447'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:39.330084 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-a20780ba-580b-4494-a269-57d042e2ea34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '611152a8389a436884553fbaba6b7447'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:39.338780 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 934/1877] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jul 15 19:36:38 2022] DELETE /identity/v3/users/611152a8389a436884553fbaba6b7447 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:36:39.990991 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ee7c89-62e1-4caa-aa69-e8ae54fee5d7 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:39.992403 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-f5ee7c89-62e1-4caa-aa69-e8ae54fee5d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:36:40.009974 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ee7c89-62e1-4caa-aa69-e8ae54fee5d7 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:40.074408 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02263aaf-71f1-4435-a47a-dee922e431c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:40.075542 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02263aaf-71f1-4435-a47a-dee922e431c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:40.076013 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02263aaf-71f1-4435-a47a-dee922e431c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:40.076329 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02263aaf-71f1-4435-a47a-dee922e431c7 admin admin] PATH_INFO: `/v3/projects/a9b79cb0f5004bbd91601bd1e4ddf892` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:40.077523 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02263aaf-71f1-4435-a47a-dee922e431c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9b79cb0f5004bbd91601bd1e4ddf892)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:40.079464 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02263aaf-71f1-4435-a47a-dee922e431c7 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:40.087973 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:36:40.207421 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-02263aaf-71f1-4435-a47a-dee922e431c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9b79cb0f5004bbd91601bd1e4ddf892', 'request_id': 'req-02263aaf-71f1-4435-a47a-dee922e431c7'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:40.327628 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-02263aaf-71f1-4435-a47a-dee922e431c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9b79cb0f5004bbd91601bd1e4ddf892', 'request_id': 'req-02263aaf-71f1-4435-a47a-dee922e431c7'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:40.335147 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-02263aaf-71f1-4435-a47a-dee922e431c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9b79cb0f5004bbd91601bd1e4ddf892', 'request_id': 'req-02263aaf-71f1-4435-a47a-dee922e431c7'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:40.346680 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-02263aaf-71f1-4435-a47a-dee922e431c7 admin admin] The token cache is being invalidate because project a9b79cb0f5004bbd91601bd1e4ddf892 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:36:40.346978 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-02263aaf-71f1-4435-a47a-dee922e431c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:40.347668 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-02263aaf-71f1-4435-a47a-dee922e431c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:36:40.349901 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 944/1878] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jul 15 19:36:39 2022] DELETE /identity/v3/projects/a9b79cb0f5004bbd91601bd1e4ddf892 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:36:40.354927 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c84f7-1264-4cda-aa6d-aea3bcd739b7 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:40.355406 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c84f7-1264-4cda-aa6d-aea3bcd739b7 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:40.355486 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c84f7-1264-4cda-aa6d-aea3bcd739b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:40.384902 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-be2c84f7-1264-4cda-aa6d-aea3bcd739b7 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:36:40.386457 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-be2c84f7-1264-4cda-aa6d-aea3bcd739b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:36:40.400755 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02263aaf-71f1-4435-a47a-dee922e431c7 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:40.402089 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-02263aaf-71f1-4435-a47a-dee922e431c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:36:40.470974 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02263aaf-71f1-4435-a47a-dee922e431c7 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:40.475273 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee779855-69ae-44f5-aad8-216e25bde416 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86c2df3d378490ca7ea497a73c19d40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:40.475923 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee779855-69ae-44f5-aad8-216e25bde416 service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:40.476236 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee779855-69ae-44f5-aad8-216e25bde416 service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:40.480337 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee779855-69ae-44f5-aad8-216e25bde416 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:40.480337 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-ee779855-69ae-44f5-aad8-216e25bde416 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:36:40.480603 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 935/1879] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:36:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:36:40.487822 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1df6a5e-7b51-4998-b329-c0d883db3817 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:40.488188 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1df6a5e-7b51-4998-b329-c0d883db3817 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:40.488562 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1df6a5e-7b51-4998-b329-c0d883db3817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:40.507974 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-a1df6a5e-7b51-4998-b329-c0d883db3817 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:36:40.509106 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-a1df6a5e-7b51-4998-b329-c0d883db3817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:36:40.518199 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 936/1880] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:36:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:36:40.523414 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1df6a5e-7b51-4998-b329-c0d883db3817 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:40.531987 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1df6a5e-7b51-4998-b329-c0d883db3817 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:40.537389 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b318ea9a-fe85-4b05-8bfb-39fd8c82ebca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:40.538018 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b318ea9a-fe85-4b05-8bfb-39fd8c82ebca admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:40.538301 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b318ea9a-fe85-4b05-8bfb-39fd8c82ebca admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:40.538569 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b318ea9a-fe85-4b05-8bfb-39fd8c82ebca admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:40.539349 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b318ea9a-fe85-4b05-8bfb-39fd8c82ebca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:36:40.539658 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b318ea9a-fe85-4b05-8bfb-39fd8c82ebca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:40.541313 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b318ea9a-fe85-4b05-8bfb-39fd8c82ebca admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:40.545073 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee779855-69ae-44f5-aad8-216e25bde416 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:40.546544 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee779855-69ae-44f5-aad8-216e25bde416 service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:40.553583 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 937/1881] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jul 15 19:36:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:36:40.557135 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 945/1882] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:36:40 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 19:36:40.558102 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b318ea9a-fe85-4b05-8bfb-39fd8c82ebca admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:40.568497 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b318ea9a-fe85-4b05-8bfb-39fd8c82ebca admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:40.578168 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9761c6b3-c028-45ab-a4fa-c3e083042a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:40.578168 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9761c6b3-c028-45ab-a4fa-c3e083042a5b admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:40.578168 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9761c6b3-c028-45ab-a4fa-c3e083042a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:40.583152 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9761c6b3-c028-45ab-a4fa-c3e083042a5b admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:40.583152 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9761c6b3-c028-45ab-a4fa-c3e083042a5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:40.583152 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9761c6b3-c028-45ab-a4fa-c3e083042a5b admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:40.685443 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 938/1883] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jul 15 19:36:40 2022] POST /identity/v3/projects => generated 375 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:36:40.689689 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee779855-69ae-44f5-aad8-216e25bde416 service cinder] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:40.701845 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee779855-69ae-44f5-aad8-216e25bde416 service cinder] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:40.762390 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:36:40.776438 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72790df-555d-4c3b-ba37-9824130ecb45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:40.777450 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72790df-555d-4c3b-ba37-9824130ecb45 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:40.777972 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72790df-555d-4c3b-ba37-9824130ecb45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:40.778404 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72790df-555d-4c3b-ba37-9824130ecb45 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:40.779288 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72790df-555d-4c3b-ba37-9824130ecb45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:40.781332 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72790df-555d-4c3b-ba37-9824130ecb45 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:40.902589 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 946/1884] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jul 15 19:36:40 2022] POST /identity/v3/users => generated 288 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:36:40.906837 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9761c6b3-c028-45ab-a4fa-c3e083042a5b admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:40.918042 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9761c6b3-c028-45ab-a4fa-c3e083042a5b admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:40.922684 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bbf081-59e1-46fe-9152-cb869ead4da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:40.923389 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bbf081-59e1-46fe-9152-cb869ead4da2 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:40.923671 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bbf081-59e1-46fe-9152-cb869ead4da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:40.923948 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bbf081-59e1-46fe-9152-cb869ead4da2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:40.924681 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bbf081-59e1-46fe-9152-cb869ead4da2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:40.926427 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bbf081-59e1-46fe-9152-cb869ead4da2 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:40.949620 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 939/1885] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:36:40 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:36:40.954947 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72790df-555d-4c3b-ba37-9824130ecb45 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:40.965168 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72790df-555d-4c3b-ba37-9824130ecb45 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:40.970018 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a247e47-2afe-4687-be11-837cefc23199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:40.970719 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a247e47-2afe-4687-be11-837cefc23199 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:40.971051 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a247e47-2afe-4687-be11-837cefc23199 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:40.971327 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a247e47-2afe-4687-be11-837cefc23199 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:40.972071 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a247e47-2afe-4687-be11-837cefc23199 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:40.973704 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a247e47-2afe-4687-be11-837cefc23199 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:40.987297 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 947/1886] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:36:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:36:40.991201 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bbf081-59e1-46fe-9152-cb869ead4da2 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:41.004915 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bbf081-59e1-46fe-9152-cb869ead4da2 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:41.010044 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db84690c-4001-410b-8760-a98de810e19c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:41.011391 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84690c-4001-410b-8760-a98de810e19c admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:41.011673 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84690c-4001-410b-8760-a98de810e19c admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:41.011950 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84690c-4001-410b-8760-a98de810e19c admin admin] PATH_INFO: `/v3/projects/670a6f26f92b477f98168f4340e0176b/users/e03ecc37b14d406987ca1ffc443a0874/roles/28763c6e70094116b3a4b2895895fa58` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:41.028999 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db84690c-4001-410b-8760-a98de810e19c admin admin] RBAC: Authorizing `identity:create_grant(project_id=670a6f26f92b477f98168f4340e0176b, user_id=e03ecc37b14d406987ca1ffc443a0874, role_id=28763c6e70094116b3a4b2895895fa58)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:41.031645 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db84690c-4001-410b-8760-a98de810e19c admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:41.168296 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 940/1887] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:36:40 2022] PUT /identity/v3/projects/670a6f26f92b477f98168f4340e0176b/users/e03ecc37b14d406987ca1ffc443a0874/roles/28763c6e70094116b3a4b2895895fa58 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:36:41.180431 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bcd35b-18de-4115-9b59-06f640a4cca8 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:41.180939 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bcd35b-18de-4115-9b59-06f640a4cca8 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:41.181042 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bcd35b-18de-4115-9b59-06f640a4cca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:41.215657 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-52bcd35b-18de-4115-9b59-06f640a4cca8 None None] MFA Rules not processed for user `e03ecc37b14d406987ca1ffc443a0874`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:36:41.216898 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-52bcd35b-18de-4115-9b59-06f640a4cca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:36:41.296216 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 948/1888] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:36:41 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:36:41.300737 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db84690c-4001-410b-8760-a98de810e19c admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:41.409894 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db84690c-4001-410b-8760-a98de810e19c admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:41.416684 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d627a863-3c29-4cde-9478-730fafb1c56f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:41.417587 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d627a863-3c29-4cde-9478-730fafb1c56f admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:41.418100 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d627a863-3c29-4cde-9478-730fafb1c56f admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:41.418554 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d627a863-3c29-4cde-9478-730fafb1c56f admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:41.419480 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d627a863-3c29-4cde-9478-730fafb1c56f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:41.421591 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d627a863-3c29-4cde-9478-730fafb1c56f admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:41.434835 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:INSERT=3,keystone:DELETE=1,keystone:SELECT=96 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:36:41.546442 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 941/1889] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jul 15 19:36:41 2022] POST /identity/v3/projects => generated 375 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:36:41.546970 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bcd35b-18de-4115-9b59-06f640a4cca8 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:41.560653 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bcd35b-18de-4115-9b59-06f640a4cca8 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:41.650030 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c4177e-4d28-426e-9756-e6b4b4cdc289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:41.650773 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c4177e-4d28-426e-9756-e6b4b4cdc289 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:41.651126 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c4177e-4d28-426e-9756-e6b4b4cdc289 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:41.651473 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c4177e-4d28-426e-9756-e6b4b4cdc289 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:41.652262 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c4177e-4d28-426e-9756-e6b4b4cdc289 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:41.654649 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c4177e-4d28-426e-9756-e6b4b4cdc289 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:41.719542 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 949/1890] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jul 15 19:36:41 2022] POST /identity/v3/users => generated 294 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:36:41.727436 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d627a863-3c29-4cde-9478-730fafb1c56f admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:41.737737 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d627a863-3c29-4cde-9478-730fafb1c56f admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:41.742344 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5028e3c2-3136-45f9-b199-f6a2e2cd7dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:41.742997 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5028e3c2-3136-45f9-b199-f6a2e2cd7dab admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:41.743276 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5028e3c2-3136-45f9-b199-f6a2e2cd7dab admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:41.743547 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5028e3c2-3136-45f9-b199-f6a2e2cd7dab admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:41.744266 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5028e3c2-3136-45f9-b199-f6a2e2cd7dab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:41.745904 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5028e3c2-3136-45f9-b199-f6a2e2cd7dab admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:41.757424 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 942/1891] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:36:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:36:41.761997 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c4177e-4d28-426e-9756-e6b4b4cdc289 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:41.772180 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c4177e-4d28-426e-9756-e6b4b4cdc289 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:41.776776 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15974f98-91e8-4ce5-bf7e-58b2d7c030ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:41.777732 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15974f98-91e8-4ce5-bf7e-58b2d7c030ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:41.778047 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15974f98-91e8-4ce5-bf7e-58b2d7c030ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:41.778328 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15974f98-91e8-4ce5-bf7e-58b2d7c030ac admin admin] PATH_INFO: `/v3/projects/5837828c54804b74b96dfbfe8acb7207/users/0fb82d64c34a46ca8a469777735e2aad/roles/4318f86ba9f74cd4a0bef1c311276657` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:41.796739 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15974f98-91e8-4ce5-bf7e-58b2d7c030ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=5837828c54804b74b96dfbfe8acb7207, user_id=0fb82d64c34a46ca8a469777735e2aad, role_id=4318f86ba9f74cd4a0bef1c311276657)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:41.799356 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15974f98-91e8-4ce5-bf7e-58b2d7c030ac admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:41.866930 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 950/1892] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:36:41 2022] PUT /identity/v3/projects/5837828c54804b74b96dfbfe8acb7207/users/0fb82d64c34a46ca8a469777735e2aad/roles/4318f86ba9f74cd4a0bef1c311276657 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:36:41.870923 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5028e3c2-3136-45f9-b199-f6a2e2cd7dab admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:41.881511 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5028e3c2-3136-45f9-b199-f6a2e2cd7dab admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:41.937382 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfd24a4-f850-4a25-bbee-0d0166a46739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:41.938086 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfd24a4-f850-4a25-bbee-0d0166a46739 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:41.938376 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfd24a4-f850-4a25-bbee-0d0166a46739 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:41.938655 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfd24a4-f850-4a25-bbee-0d0166a46739 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:41.939459 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfd24a4-f850-4a25-bbee-0d0166a46739 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:41.941061 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfd24a4-f850-4a25-bbee-0d0166a46739 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:41.951694 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 943/1893] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:36:41 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 19:36:41.956019 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15974f98-91e8-4ce5-bf7e-58b2d7c030ac admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:41.966344 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15974f98-91e8-4ce5-bf7e-58b2d7c030ac admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:41.970867 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8eea9b-3943-4225-aaeb-f5076bacdbdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:41.971855 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8eea9b-3943-4225-aaeb-f5076bacdbdb admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:41.972035 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8eea9b-3943-4225-aaeb-f5076bacdbdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:41.972307 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8eea9b-3943-4225-aaeb-f5076bacdbdb admin admin] PATH_INFO: `/v3/projects/5837828c54804b74b96dfbfe8acb7207/users/0fb82d64c34a46ca8a469777735e2aad/roles/4318f86ba9f74cd4a0bef1c311276657` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:41.974858 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8eea9b-3943-4225-aaeb-f5076bacdbdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=5837828c54804b74b96dfbfe8acb7207, user_id=0fb82d64c34a46ca8a469777735e2aad, role_id=4318f86ba9f74cd4a0bef1c311276657)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:41.977506 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8eea9b-3943-4225-aaeb-f5076bacdbdb admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:41.988139 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 951/1894] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:36:41 2022] PUT /identity/v3/projects/5837828c54804b74b96dfbfe8acb7207/users/0fb82d64c34a46ca8a469777735e2aad/roles/4318f86ba9f74cd4a0bef1c311276657 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:36:42.002926 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1729c875-f216-4a89-96be-659f69832a4f None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:42.003222 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1729c875-f216-4a89-96be-659f69832a4f None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:42.003506 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1729c875-f216-4a89-96be-659f69832a4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:42.030446 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-1729c875-f216-4a89-96be-659f69832a4f None None] MFA Rules not processed for user `0fb82d64c34a46ca8a469777735e2aad`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:36:42.031691 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-1729c875-f216-4a89-96be-659f69832a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:36:42.110552 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 944/1895] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:36:42 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:36:42.123032 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8eea9b-3943-4225-aaeb-f5076bacdbdb admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:42.124701 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-9e8eea9b-3943-4225-aaeb-f5076bacdbdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:36:42.134802 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:36:42.217815 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8eea9b-3943-4225-aaeb-f5076bacdbdb admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:42.222441 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1942a7-3a42-4b0d-bc50-0ce44be946cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86c2df3d378490ca7ea497a73c19d40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:42.223637 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1942a7-3a42-4b0d-bc50-0ce44be946cf service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:42.223968 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1942a7-3a42-4b0d-bc50-0ce44be946cf service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:42.224249 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1942a7-3a42-4b0d-bc50-0ce44be946cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:42.313750 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1942a7-3a42-4b0d-bc50-0ce44be946cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:42.315232 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1942a7-3a42-4b0d-bc50-0ce44be946cf service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:42.328023 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 952/1896] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:36:42 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:36:52.107309 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=71,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:36:52.321922 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=45 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:37:07.794309 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1729c875-f216-4a89-96be-659f69832a4f None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:07.804202 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1729c875-f216-4a89-96be-659f69832a4f None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:07.807790 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990df820-25bd-4577-8e78-064cd26abd1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86c2df3d378490ca7ea497a73c19d40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:07.808441 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990df820-25bd-4577-8e78-064cd26abd1d service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:07.808725 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990df820-25bd-4577-8e78-064cd26abd1d service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:07.822224 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990df820-25bd-4577-8e78-064cd26abd1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:07.822224 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990df820-25bd-4577-8e78-064cd26abd1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:07.822224 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990df820-25bd-4577-8e78-064cd26abd1d service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:07.832872 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 945/1897] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:37:07 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:37:17.825676 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=6 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:37:19.574330 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1942a7-3a42-4b0d-bc50-0ce44be946cf service cinder] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:19.577331 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-0e1942a7-3a42-4b0d-bc50-0ce44be946cf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:37:19.681036 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1942a7-3a42-4b0d-bc50-0ce44be946cf service cinder] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:19.687139 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80284f0f-13fe-4017-8294-5b6d1a3936e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:19.688571 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80284f0f-13fe-4017-8294-5b6d1a3936e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:19.689166 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80284f0f-13fe-4017-8294-5b6d1a3936e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:19.690260 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80284f0f-13fe-4017-8294-5b6d1a3936e1 admin admin] PATH_INFO: `/v3/users/8672e699bb1d41a68932b39ba176b404` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:19.692689 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80284f0f-13fe-4017-8294-5b6d1a3936e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8672e699bb1d41a68932b39ba176b404)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:19.695414 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80284f0f-13fe-4017-8294-5b6d1a3936e1 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:19.980726 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-80284f0f-13fe-4017-8294-5b6d1a3936e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8672e699bb1d41a68932b39ba176b404'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:19.993755 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-80284f0f-13fe-4017-8294-5b6d1a3936e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8672e699bb1d41a68932b39ba176b404'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:20.083132 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-80284f0f-13fe-4017-8294-5b6d1a3936e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8672e699bb1d41a68932b39ba176b404'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:20.083771 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-80284f0f-13fe-4017-8294-5b6d1a3936e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8672e699bb1d41a68932b39ba176b404'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:20.084393 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-80284f0f-13fe-4017-8294-5b6d1a3936e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8672e699bb1d41a68932b39ba176b404'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:20.107401 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e374d2-476d-4219-abf2-6d9c4ff7cdff None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:20.108366 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e374d2-476d-4219-abf2-6d9c4ff7cdff None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:20.108882 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e374d2-476d-4219-abf2-6d9c4ff7cdff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:20.109864 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 953/1898] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jul 15 19:37:19 2022] DELETE /identity/v3/users/8672e699bb1d41a68932b39ba176b404 => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:37:20.129386 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-f8e374d2-476d-4219-abf2-6d9c4ff7cdff None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:37:20.130767 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-f8e374d2-476d-4219-abf2-6d9c4ff7cdff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:37:20.229813 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 946/1899] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:37:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:37:20.253176 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80284f0f-13fe-4017-8294-5b6d1a3936e1 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:20.253176 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-80284f0f-13fe-4017-8294-5b6d1a3936e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:37:20.362672 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80284f0f-13fe-4017-8294-5b6d1a3936e1 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:20.369678 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577e2e1d-6e89-4494-8a69-55d927e18f64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:20.379248 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577e2e1d-6e89-4494-8a69-55d927e18f64 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:20.379248 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577e2e1d-6e89-4494-8a69-55d927e18f64 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:20.379248 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577e2e1d-6e89-4494-8a69-55d927e18f64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:20.386196 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577e2e1d-6e89-4494-8a69-55d927e18f64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:20.388004 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577e2e1d-6e89-4494-8a69-55d927e18f64 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:20.407044 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 954/1900] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:37:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:37:20.783187 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e374d2-476d-4219-abf2-6d9c4ff7cdff None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:20.785149 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-f8e374d2-476d-4219-abf2-6d9c4ff7cdff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:37:20.798135 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e374d2-476d-4219-abf2-6d9c4ff7cdff None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:20.809363 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebdcf42-0b0a-4a5c-827f-99ae8d501e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:20.813522 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebdcf42-0b0a-4a5c-827f-99ae8d501e24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:20.813522 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebdcf42-0b0a-4a5c-827f-99ae8d501e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:20.814183 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebdcf42-0b0a-4a5c-827f-99ae8d501e24 admin admin] PATH_INFO: `/v3/users/5dc28ae2cadf44e596f41eee5ae3bdc0` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:20.817129 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebdcf42-0b0a-4a5c-827f-99ae8d501e24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5dc28ae2cadf44e596f41eee5ae3bdc0)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:20.819458 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebdcf42-0b0a-4a5c-827f-99ae8d501e24 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:21.028218 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-0ebdcf42-0b0a-4a5c-827f-99ae8d501e24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5dc28ae2cadf44e596f41eee5ae3bdc0'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:21.043340 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-0ebdcf42-0b0a-4a5c-827f-99ae8d501e24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5dc28ae2cadf44e596f41eee5ae3bdc0'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:21.128808 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-0ebdcf42-0b0a-4a5c-827f-99ae8d501e24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5dc28ae2cadf44e596f41eee5ae3bdc0'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:21.129525 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-0ebdcf42-0b0a-4a5c-827f-99ae8d501e24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5dc28ae2cadf44e596f41eee5ae3bdc0'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:21.138909 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-0ebdcf42-0b0a-4a5c-827f-99ae8d501e24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5dc28ae2cadf44e596f41eee5ae3bdc0'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:21.148707 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 947/1901] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jul 15 19:37:20 2022] DELETE /identity/v3/users/5dc28ae2cadf44e596f41eee5ae3bdc0 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:37:21.150786 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79e8836-9331-4fdb-8c42-fcc58fe92958 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:21.151349 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79e8836-9331-4fdb-8c42-fcc58fe92958 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:21.151349 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79e8836-9331-4fdb-8c42-fcc58fe92958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:21.170216 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-e79e8836-9331-4fdb-8c42-fcc58fe92958 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:37:21.171505 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-e79e8836-9331-4fdb-8c42-fcc58fe92958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:37:21.190825 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:37:21.203733 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebdcf42-0b0a-4a5c-827f-99ae8d501e24 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:21.218021 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0ebdcf42-0b0a-4a5c-827f-99ae8d501e24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:37:21.248097 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 955/1902] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:37:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:37:21.258451 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79e8836-9331-4fdb-8c42-fcc58fe92958 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:21.260926 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-e79e8836-9331-4fdb-8c42-fcc58fe92958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:37:21.335974 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79e8836-9331-4fdb-8c42-fcc58fe92958 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:21.340204 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec10d700-a12f-4263-b9ae-9d593076c7c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:21.340967 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec10d700-a12f-4263-b9ae-9d593076c7c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:21.341360 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec10d700-a12f-4263-b9ae-9d593076c7c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:21.342053 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec10d700-a12f-4263-b9ae-9d593076c7c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:21.348276 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebdcf42-0b0a-4a5c-827f-99ae8d501e24 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:21.353218 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0f606f-4b3a-472f-8746-f85958f5829a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:21.354217 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0f606f-4b3a-472f-8746-f85958f5829a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:21.354572 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0f606f-4b3a-472f-8746-f85958f5829a admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:21.354967 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0f606f-4b3a-472f-8746-f85958f5829a admin admin] PATH_INFO: `/v3/projects/52b7f5b1548749708474ce058ee4091a` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:21.356924 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0f606f-4b3a-472f-8746-f85958f5829a admin admin] RBAC: Authorizing `identity:delete_project(project_id=52b7f5b1548749708474ce058ee4091a)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:21.358425 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0f606f-4b3a-472f-8746-f85958f5829a admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:21.359763 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec10d700-a12f-4263-b9ae-9d593076c7c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:21.362084 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec10d700-a12f-4263-b9ae-9d593076c7c0 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:21.374764 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 956/1903] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:37:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 19:37:21.385679 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:37:21.643814 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-2f0f606f-4b3a-472f-8746-f85958f5829a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52b7f5b1548749708474ce058ee4091a', 'request_id': 'req-2f0f606f-4b3a-472f-8746-f85958f5829a'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:21.757065 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-2f0f606f-4b3a-472f-8746-f85958f5829a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52b7f5b1548749708474ce058ee4091a', 'request_id': 'req-2f0f606f-4b3a-472f-8746-f85958f5829a'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:21.765820 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-2f0f606f-4b3a-472f-8746-f85958f5829a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52b7f5b1548749708474ce058ee4091a', 'request_id': 'req-2f0f606f-4b3a-472f-8746-f85958f5829a'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:21.795214 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-2f0f606f-4b3a-472f-8746-f85958f5829a admin admin] The token cache is being invalidate because project 52b7f5b1548749708474ce058ee4091a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:37:21.795623 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-2f0f606f-4b3a-472f-8746-f85958f5829a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:21.796356 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-2f0f606f-4b3a-472f-8746-f85958f5829a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:21.798328 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 948/1904] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jul 15 19:37:21 2022] DELETE /identity/v3/projects/52b7f5b1548749708474ce058ee4091a => generated 0 bytes in 595 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:37:21.805899 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec10d700-a12f-4263-b9ae-9d593076c7c0 service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:21.816885 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec10d700-a12f-4263-b9ae-9d593076c7c0 service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:21.876874 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4881d43f-386f-4e59-b2a7-3b6e4e348ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:21.877821 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4881d43f-386f-4e59-b2a7-3b6e4e348ff8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:21.878109 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4881d43f-386f-4e59-b2a7-3b6e4e348ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:21.878418 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4881d43f-386f-4e59-b2a7-3b6e4e348ff8 admin admin] PATH_INFO: `/v3/users/c32ae423848f42049d07ee43c4fd2c1e` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:21.879649 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4881d43f-386f-4e59-b2a7-3b6e4e348ff8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c32ae423848f42049d07ee43c4fd2c1e)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:21.881706 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4881d43f-386f-4e59-b2a7-3b6e4e348ff8 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:21.890974 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0f606f-4b3a-472f-8746-f85958f5829a admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:21.892374 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-2f0f606f-4b3a-472f-8746-f85958f5829a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:37:21.928802 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0f606f-4b3a-472f-8746-f85958f5829a admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:21.934363 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f465853-b6c0-4ffa-ad57-307e3cd7bf53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:21.935251 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f465853-b6c0-4ffa-ad57-307e3cd7bf53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:21.935534 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f465853-b6c0-4ffa-ad57-307e3cd7bf53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:21.935813 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f465853-b6c0-4ffa-ad57-307e3cd7bf53 admin admin] PATH_INFO: `/v3/projects/ce489d0b15524c3ebb161ac10ec1a7c3` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:21.937083 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f465853-b6c0-4ffa-ad57-307e3cd7bf53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce489d0b15524c3ebb161ac10ec1a7c3)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:21.939068 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f465853-b6c0-4ffa-ad57-307e3cd7bf53 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:22.185491 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4f465853-b6c0-4ffa-ad57-307e3cd7bf53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce489d0b15524c3ebb161ac10ec1a7c3', 'request_id': 'req-4f465853-b6c0-4ffa-ad57-307e3cd7bf53'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:22.190349 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-4881d43f-386f-4e59-b2a7-3b6e4e348ff8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c32ae423848f42049d07ee43c4fd2c1e'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:22.206912 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-4881d43f-386f-4e59-b2a7-3b6e4e348ff8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c32ae423848f42049d07ee43c4fd2c1e'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:22.210049 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:37:22.288818 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4f465853-b6c0-4ffa-ad57-307e3cd7bf53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce489d0b15524c3ebb161ac10ec1a7c3', 'request_id': 'req-4f465853-b6c0-4ffa-ad57-307e3cd7bf53'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:22.301804 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4f465853-b6c0-4ffa-ad57-307e3cd7bf53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce489d0b15524c3ebb161ac10ec1a7c3', 'request_id': 'req-4f465853-b6c0-4ffa-ad57-307e3cd7bf53'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:22.309465 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4f465853-b6c0-4ffa-ad57-307e3cd7bf53 admin admin] The token cache is being invalidate because project ce489d0b15524c3ebb161ac10ec1a7c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:37:22.309757 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4f465853-b6c0-4ffa-ad57-307e3cd7bf53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:22.310569 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4f465853-b6c0-4ffa-ad57-307e3cd7bf53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:22.312135 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 949/1905] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jul 15 19:37:21 2022] DELETE /identity/v3/projects/ce489d0b15524c3ebb161ac10ec1a7c3 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:37:22.316760 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f465853-b6c0-4ffa-ad57-307e3cd7bf53 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:22.318732 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-4f465853-b6c0-4ffa-ad57-307e3cd7bf53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:37:22.387263 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f465853-b6c0-4ffa-ad57-307e3cd7bf53 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:22.391817 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-4881d43f-386f-4e59-b2a7-3b6e4e348ff8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c32ae423848f42049d07ee43c4fd2c1e'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:22.392114 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-4881d43f-386f-4e59-b2a7-3b6e4e348ff8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c32ae423848f42049d07ee43c4fd2c1e'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:22.392603 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e98cb9-af7b-4fb2-8c7c-786ed8403638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:22.392947 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-4881d43f-386f-4e59-b2a7-3b6e4e348ff8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c32ae423848f42049d07ee43c4fd2c1e'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:22.393599 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e98cb9-af7b-4fb2-8c7c-786ed8403638 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:22.393918 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e98cb9-af7b-4fb2-8c7c-786ed8403638 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:22.394198 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e98cb9-af7b-4fb2-8c7c-786ed8403638 admin admin] PATH_INFO: `/v3/users/d28e0716521042688a4f697005069925` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:22.395477 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e98cb9-af7b-4fb2-8c7c-786ed8403638 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d28e0716521042688a4f697005069925)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:22.397413 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e98cb9-af7b-4fb2-8c7c-786ed8403638 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:22.416585 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 957/1906] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jul 15 19:37:21 2022] DELETE /identity/v3/users/c32ae423848f42049d07ee43c4fd2c1e => generated 0 bytes in 612 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:37:22.593929 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:37:22.717799 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-86e98cb9-af7b-4fb2-8c7c-786ed8403638 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd28e0716521042688a4f697005069925'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:22.744182 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-86e98cb9-af7b-4fb2-8c7c-786ed8403638 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd28e0716521042688a4f697005069925'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:22.869031 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-86e98cb9-af7b-4fb2-8c7c-786ed8403638 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd28e0716521042688a4f697005069925'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:22.869675 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-86e98cb9-af7b-4fb2-8c7c-786ed8403638 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd28e0716521042688a4f697005069925'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:22.878010 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-86e98cb9-af7b-4fb2-8c7c-786ed8403638 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd28e0716521042688a4f697005069925'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:22.884963 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 950/1907] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jul 15 19:37:22 2022] DELETE /identity/v3/users/d28e0716521042688a4f697005069925 => generated 0 bytes in 569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:37:23.017840 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4881d43f-386f-4e59-b2a7-3b6e4e348ff8 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:23.019498 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-4881d43f-386f-4e59-b2a7-3b6e4e348ff8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:37:23.116544 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4881d43f-386f-4e59-b2a7-3b6e4e348ff8 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:23.175731 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d126523-179b-49ac-b4c1-3ece76cd89a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:23.176626 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d126523-179b-49ac-b4c1-3ece76cd89a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:23.176910 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d126523-179b-49ac-b4c1-3ece76cd89a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:23.177191 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d126523-179b-49ac-b4c1-3ece76cd89a1 admin admin] PATH_INFO: `/v3/projects/cfcac46621424b08a8fb141a5d5bf736` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:23.178484 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d126523-179b-49ac-b4c1-3ece76cd89a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfcac46621424b08a8fb141a5d5bf736)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:23.180493 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d126523-179b-49ac-b4c1-3ece76cd89a1 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:23.324454 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-4d126523-179b-49ac-b4c1-3ece76cd89a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfcac46621424b08a8fb141a5d5bf736', 'request_id': 'req-4d126523-179b-49ac-b4c1-3ece76cd89a1'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:23.426430 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-4d126523-179b-49ac-b4c1-3ece76cd89a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfcac46621424b08a8fb141a5d5bf736', 'request_id': 'req-4d126523-179b-49ac-b4c1-3ece76cd89a1'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:23.433283 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-4d126523-179b-49ac-b4c1-3ece76cd89a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfcac46621424b08a8fb141a5d5bf736', 'request_id': 'req-4d126523-179b-49ac-b4c1-3ece76cd89a1'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:23.443601 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-4d126523-179b-49ac-b4c1-3ece76cd89a1 admin admin] The token cache is being invalidate because project cfcac46621424b08a8fb141a5d5bf736 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:37:23.443888 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-4d126523-179b-49ac-b4c1-3ece76cd89a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:23.444536 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-4d126523-179b-49ac-b4c1-3ece76cd89a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:23.446134 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 958/1908] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jul 15 19:37:23 2022] DELETE /identity/v3/projects/cfcac46621424b08a8fb141a5d5bf736 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:37:23.452338 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57e6a33-501f-4cb4-b3b8-44b9ae84daf2 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:23.452798 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57e6a33-501f-4cb4-b3b8-44b9ae84daf2 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:23.452887 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57e6a33-501f-4cb4-b3b8-44b9ae84daf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:23.470274 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-d57e6a33-501f-4cb4-b3b8-44b9ae84daf2 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:37:23.476445 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d57e6a33-501f-4cb4-b3b8-44b9ae84daf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:37:23.520874 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d126523-179b-49ac-b4c1-3ece76cd89a1 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:23.528292 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-4d126523-179b-49ac-b4c1-3ece76cd89a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:37:23.560187 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 951/1909] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:37:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:37:23.560557 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:37:23.566064 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be62c8-f165-4fc5-a664-dbbc9f6053ff None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:23.566364 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be62c8-f165-4fc5-a664-dbbc9f6053ff None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:23.566624 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be62c8-f165-4fc5-a664-dbbc9f6053ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:23.586287 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-c1be62c8-f165-4fc5-a664-dbbc9f6053ff None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:37:23.588108 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-c1be62c8-f165-4fc5-a664-dbbc9f6053ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:37:23.596949 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 952/1910] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:37:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:37:23.601334 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1be62c8-f165-4fc5-a664-dbbc9f6053ff None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:23.606826 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d126523-179b-49ac-b4c1-3ece76cd89a1 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:23.610453 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1be62c8-f165-4fc5-a664-dbbc9f6053ff None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:23.612501 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810d4f97-4fae-47a0-9e02-aa7799d7aaa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:23.613963 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d4f97-4fae-47a0-9e02-aa7799d7aaa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:23.614443 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d4f97-4fae-47a0-9e02-aa7799d7aaa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:23.614913 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d4f97-4fae-47a0-9e02-aa7799d7aaa7 admin admin] PATH_INFO: `/v3/projects/fa570668a1b144ebb5dd90965d2d45aa` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:23.616798 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810d4f97-4fae-47a0-9e02-aa7799d7aaa7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa570668a1b144ebb5dd90965d2d45aa)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:23.617303 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24a5837-5270-46b5-a9b5-e16d0f4e86ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:23.618326 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24a5837-5270-46b5-a9b5-e16d0f4e86ab admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:23.618816 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24a5837-5270-46b5-a9b5-e16d0f4e86ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:23.619274 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24a5837-5270-46b5-a9b5-e16d0f4e86ab admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:23.620247 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810d4f97-4fae-47a0-9e02-aa7799d7aaa7 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:23.620686 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24a5837-5270-46b5-a9b5-e16d0f4e86ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:37:23.621181 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24a5837-5270-46b5-a9b5-e16d0f4e86ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:23.624003 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24a5837-5270-46b5-a9b5-e16d0f4e86ab admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:23.643424 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 953/1911] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jul 15 19:37:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:37:23.646788 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24a5837-5270-46b5-a9b5-e16d0f4e86ab admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:23.657508 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24a5837-5270-46b5-a9b5-e16d0f4e86ab admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:23.664110 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6b9ee6-f005-49dd-8070-a8372962fefd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:23.664806 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6b9ee6-f005-49dd-8070-a8372962fefd admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:23.665114 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6b9ee6-f005-49dd-8070-a8372962fefd admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:23.665523 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6b9ee6-f005-49dd-8070-a8372962fefd admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:23.666262 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6b9ee6-f005-49dd-8070-a8372962fefd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:23.668290 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6b9ee6-f005-49dd-8070-a8372962fefd admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:23.791968 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-810d4f97-4fae-47a0-9e02-aa7799d7aaa7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa570668a1b144ebb5dd90965d2d45aa', 'request_id': 'req-810d4f97-4fae-47a0-9e02-aa7799d7aaa7'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:23.798111 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 954/1912] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jul 15 19:37:23 2022] POST /identity/v3/projects => generated 359 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:37:23.802631 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6b9ee6-f005-49dd-8070-a8372962fefd admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:23.869367 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6b9ee6-f005-49dd-8070-a8372962fefd admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:23.874120 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80776233-79fb-48a8-b393-ab5424977c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:23.874814 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80776233-79fb-48a8-b393-ab5424977c58 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:23.875114 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80776233-79fb-48a8-b393-ab5424977c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:23.875396 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80776233-79fb-48a8-b393-ab5424977c58 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:23.876089 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80776233-79fb-48a8-b393-ab5424977c58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:23.878085 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80776233-79fb-48a8-b393-ab5424977c58 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:23.887423 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:37:23.979790 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-810d4f97-4fae-47a0-9e02-aa7799d7aaa7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa570668a1b144ebb5dd90965d2d45aa', 'request_id': 'req-810d4f97-4fae-47a0-9e02-aa7799d7aaa7'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:23.985975 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 955/1913] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jul 15 19:37:23 2022] POST /identity/v3/users => generated 280 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:37:23.987142 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-810d4f97-4fae-47a0-9e02-aa7799d7aaa7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa570668a1b144ebb5dd90965d2d45aa', 'request_id': 'req-810d4f97-4fae-47a0-9e02-aa7799d7aaa7'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:23.990393 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80776233-79fb-48a8-b393-ab5424977c58 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:23.997931 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-810d4f97-4fae-47a0-9e02-aa7799d7aaa7 admin admin] The token cache is being invalidate because project fa570668a1b144ebb5dd90965d2d45aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:37:23.998247 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-810d4f97-4fae-47a0-9e02-aa7799d7aaa7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:23.998874 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-810d4f97-4fae-47a0-9e02-aa7799d7aaa7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:24.000050 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80776233-79fb-48a8-b393-ab5424977c58 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:24.000835 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 959/1914] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jul 15 19:37:23 2022] DELETE /identity/v3/projects/fa570668a1b144ebb5dd90965d2d45aa => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:37:24.005710 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbf3953-7c22-4fac-b989-258450167471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:24.006426 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbf3953-7c22-4fac-b989-258450167471 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:24.006711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbf3953-7c22-4fac-b989-258450167471 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:24.006956 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c76bf-89b7-48f8-8d43-88facf50ea4a None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:24.007064 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbf3953-7c22-4fac-b989-258450167471 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:24.007266 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c76bf-89b7-48f8-8d43-88facf50ea4a None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:24.007542 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c76bf-89b7-48f8-8d43-88facf50ea4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:24.007706 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbf3953-7c22-4fac-b989-258450167471 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:24.009324 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbf3953-7c22-4fac-b989-258450167471 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:24.020842 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 956/1915] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:37:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:37:24.024353 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbf3953-7c22-4fac-b989-258450167471 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:24.026070 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-efbf3953-7c22-4fac-b989-258450167471 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:37:24.026799 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-291c76bf-89b7-48f8-8d43-88facf50ea4a None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:37:24.027997 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-291c76bf-89b7-48f8-8d43-88facf50ea4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:37:24.036731 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 960/1916] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:37:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:37:24.044409 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbf3953-7c22-4fac-b989-258450167471 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:24.044755 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f997e081-b0c1-488f-bec9-06387da5c0bf None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:24.045050 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f997e081-b0c1-488f-bec9-06387da5c0bf None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:24.045341 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f997e081-b0c1-488f-bec9-06387da5c0bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:24.049026 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f93230-a1d5-4c6e-a8cb-29d1af0d50f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:24.049876 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f93230-a1d5-4c6e-a8cb-29d1af0d50f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:24.050163 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f93230-a1d5-4c6e-a8cb-29d1af0d50f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:24.050445 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f93230-a1d5-4c6e-a8cb-29d1af0d50f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:24.051174 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f93230-a1d5-4c6e-a8cb-29d1af0d50f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:24.052785 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f93230-a1d5-4c6e-a8cb-29d1af0d50f5 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:24.063126 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-f997e081-b0c1-488f-bec9-06387da5c0bf None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:37:24.064181 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 957/1917] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:37:24 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 19:37:24.064339 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-f997e081-b0c1-488f-bec9-06387da5c0bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:37:24.068273 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f93230-a1d5-4c6e-a8cb-29d1af0d50f5 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:24.072636 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 961/1918] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:37:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:37:24.078222 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f93230-a1d5-4c6e-a8cb-29d1af0d50f5 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:24.079795 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f997e081-b0c1-488f-bec9-06387da5c0bf None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:24.083628 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02306f4-dbc8-433c-a960-06e0e2d9ccef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:24.084521 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02306f4-dbc8-433c-a960-06e0e2d9ccef admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:24.084803 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02306f4-dbc8-433c-a960-06e0e2d9ccef admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:24.085077 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02306f4-dbc8-433c-a960-06e0e2d9ccef admin admin] PATH_INFO: `/v3/projects/6fe7daeb43f1454b89181e5d5fbaa351/users/8de4c0a0aca04ddaabe25abc94ef112d/roles/28763c6e70094116b3a4b2895895fa58` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:24.089379 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f997e081-b0c1-488f-bec9-06387da5c0bf None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:24.094275 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8e3df7-5088-4565-a35e-d26a7d51628d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:24.094785 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8e3df7-5088-4565-a35e-d26a7d51628d admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:24.095126 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8e3df7-5088-4565-a35e-d26a7d51628d admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:24.095296 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8e3df7-5088-4565-a35e-d26a7d51628d admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:24.096585 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8e3df7-5088-4565-a35e-d26a7d51628d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:37:24.096894 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8e3df7-5088-4565-a35e-d26a7d51628d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:24.098543 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8e3df7-5088-4565-a35e-d26a7d51628d admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:24.100214 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02306f4-dbc8-433c-a960-06e0e2d9ccef admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fe7daeb43f1454b89181e5d5fbaa351, user_id=8de4c0a0aca04ddaabe25abc94ef112d, role_id=28763c6e70094116b3a4b2895895fa58)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:24.102905 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02306f4-dbc8-433c-a960-06e0e2d9ccef admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:24.110419 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 962/1919] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jul 15 19:37:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:37:24.114755 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8e3df7-5088-4565-a35e-d26a7d51628d admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:24.124385 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8e3df7-5088-4565-a35e-d26a7d51628d admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:24.128994 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ded17c-1308-4c23-b06b-8845de647acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:24.129626 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ded17c-1308-4c23-b06b-8845de647acc admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:24.130018 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ded17c-1308-4c23-b06b-8845de647acc admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:24.130297 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ded17c-1308-4c23-b06b-8845de647acc admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:24.131982 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ded17c-1308-4c23-b06b-8845de647acc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:24.133891 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ded17c-1308-4c23-b06b-8845de647acc admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:24.202857 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 958/1920] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:37:24 2022] PUT /identity/v3/projects/6fe7daeb43f1454b89181e5d5fbaa351/users/8de4c0a0aca04ddaabe25abc94ef112d/roles/28763c6e70094116b3a4b2895895fa58 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:37:24.214400 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0425003-4fe2-45b2-aa5d-d6566f615b53 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:24.214688 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0425003-4fe2-45b2-aa5d-d6566f615b53 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:24.214964 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0425003-4fe2-45b2-aa5d-d6566f615b53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:24.238773 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 963/1921] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jul 15 19:37:24 2022] POST /identity/v3/projects => generated 383 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:37:24.242750 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-c0425003-4fe2-45b2-aa5d-d6566f615b53 None None] MFA Rules not processed for user `8de4c0a0aca04ddaabe25abc94ef112d`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:37:24.243979 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-c0425003-4fe2-45b2-aa5d-d6566f615b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:37:24.245780 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ded17c-1308-4c23-b06b-8845de647acc admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:24.318654 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 959/1922] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:37:24 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:37:24.318861 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ded17c-1308-4c23-b06b-8845de647acc admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:24.323114 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691d2ced-26f3-4223-8f05-a8e414379aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:24.323114 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d2ced-26f3-4223-8f05-a8e414379aeb admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:24.323314 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d2ced-26f3-4223-8f05-a8e414379aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:24.323534 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d2ced-26f3-4223-8f05-a8e414379aeb admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:24.324235 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691d2ced-26f3-4223-8f05-a8e414379aeb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:24.328369 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691d2ced-26f3-4223-8f05-a8e414379aeb admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:24.346731 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:37:24.359353 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0425003-4fe2-45b2-aa5d-d6566f615b53 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:24.361020 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-c0425003-4fe2-45b2-aa5d-d6566f615b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:37:24.420435 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0425003-4fe2-45b2-aa5d-d6566f615b53 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:24.424245 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e99927-a030-476d-954e-5033f744b57c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86c2df3d378490ca7ea497a73c19d40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:24.424976 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e99927-a030-476d-954e-5033f744b57c service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:24.425225 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e99927-a030-476d-954e-5033f744b57c service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:24.425615 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e99927-a030-476d-954e-5033f744b57c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:24.434531 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e99927-a030-476d-954e-5033f744b57c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:24.436010 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e99927-a030-476d-954e-5033f744b57c service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:24.445395 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 960/1923] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:37:24 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:37:24.459847 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e99927-a030-476d-954e-5033f744b57c service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:24.470367 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e99927-a030-476d-954e-5033f744b57c service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:24.474795 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 964/1924] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jul 15 19:37:24 2022] POST /identity/v3/users => generated 292 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:37:24.475228 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b262669-5308-4fe7-9b15-89c9b5807479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:24.476011 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691d2ced-26f3-4223-8f05-a8e414379aeb admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:24.476132 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b262669-5308-4fe7-9b15-89c9b5807479 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:24.476377 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b262669-5308-4fe7-9b15-89c9b5807479 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:24.476652 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b262669-5308-4fe7-9b15-89c9b5807479 admin admin] PATH_INFO: `/v3/users/8de4c0a0aca04ddaabe25abc94ef112d` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:24.477886 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b262669-5308-4fe7-9b15-89c9b5807479 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8de4c0a0aca04ddaabe25abc94ef112d)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:24.479792 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b262669-5308-4fe7-9b15-89c9b5807479 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:24.485900 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691d2ced-26f3-4223-8f05-a8e414379aeb admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:24.490483 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4293ff-950c-4b7c-84b2-15924bf1c686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:24.491130 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4293ff-950c-4b7c-84b2-15924bf1c686 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:24.491408 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4293ff-950c-4b7c-84b2-15924bf1c686 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:24.491682 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4293ff-950c-4b7c-84b2-15924bf1c686 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:24.492395 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4293ff-950c-4b7c-84b2-15924bf1c686 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:24.494002 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4293ff-950c-4b7c-84b2-15924bf1c686 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:24.504831 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 965/1925] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:37:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 19:37:24.508769 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4293ff-950c-4b7c-84b2-15924bf1c686 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:24.519555 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4293ff-950c-4b7c-84b2-15924bf1c686 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:24.524522 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5242a3e8-ee19-42b4-a90c-92f286dc4192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:24.525395 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5242a3e8-ee19-42b4-a90c-92f286dc4192 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:24.525794 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5242a3e8-ee19-42b4-a90c-92f286dc4192 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:24.526104 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5242a3e8-ee19-42b4-a90c-92f286dc4192 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:24.526901 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5242a3e8-ee19-42b4-a90c-92f286dc4192 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:24.528951 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5242a3e8-ee19-42b4-a90c-92f286dc4192 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:24.541547 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 966/1926] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:37:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 19:37:24.544508 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5242a3e8-ee19-42b4-a90c-92f286dc4192 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:24.554072 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5242a3e8-ee19-42b4-a90c-92f286dc4192 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:24.561891 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9e4d2f-7f4d-4eff-9a02-1345c09eb6c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:24.562760 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9e4d2f-7f4d-4eff-9a02-1345c09eb6c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:24.563048 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9e4d2f-7f4d-4eff-9a02-1345c09eb6c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:24.563315 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9e4d2f-7f4d-4eff-9a02-1345c09eb6c7 admin admin] PATH_INFO: `/v3/projects/392cba1373bf42a39dc647c3421031c0/users/100d42d218574b6ebc2a551e8dd3f482/roles/28763c6e70094116b3a4b2895895fa58` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:24.578999 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9e4d2f-7f4d-4eff-9a02-1345c09eb6c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=392cba1373bf42a39dc647c3421031c0, user_id=100d42d218574b6ebc2a551e8dd3f482, role_id=28763c6e70094116b3a4b2895895fa58)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:24.581756 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9e4d2f-7f4d-4eff-9a02-1345c09eb6c7 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:24.695235 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 967/1927] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:37:24 2022] PUT /identity/v3/projects/392cba1373bf42a39dc647c3421031c0/users/100d42d218574b6ebc2a551e8dd3f482/roles/28763c6e70094116b3a4b2895895fa58 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:37:24.706749 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb8c3f9-6ac2-4989-ba21-0b0768c07134 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:24.707036 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb8c3f9-6ac2-4989-ba21-0b0768c07134 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:24.707336 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb8c3f9-6ac2-4989-ba21-0b0768c07134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:24.714317 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:INSERT=2,keystone:SELECT=93,keystone:DELETE=5 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:37:24.727580 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-8b262669-5308-4fe7-9b15-89c9b5807479 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8de4c0a0aca04ddaabe25abc94ef112d'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:24.735819 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-6fb8c3f9-6ac2-4989-ba21-0b0768c07134 None None] MFA Rules not processed for user `100d42d218574b6ebc2a551e8dd3f482`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:37:24.740978 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-8b262669-5308-4fe7-9b15-89c9b5807479 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8de4c0a0aca04ddaabe25abc94ef112d'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:24.749932 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-6fb8c3f9-6ac2-4989-ba21-0b0768c07134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:37:24.814368 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-8b262669-5308-4fe7-9b15-89c9b5807479 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8de4c0a0aca04ddaabe25abc94ef112d'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:24.816333 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-8b262669-5308-4fe7-9b15-89c9b5807479 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8de4c0a0aca04ddaabe25abc94ef112d'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:24.826336 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-8b262669-5308-4fe7-9b15-89c9b5807479 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8de4c0a0aca04ddaabe25abc94ef112d'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:24.838790 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 961/1928] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jul 15 19:37:24 2022] DELETE /identity/v3/users/8de4c0a0aca04ddaabe25abc94ef112d => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:37:24.850715 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa888a3-6d64-4a92-800b-6036dd6e8c8d None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:24.850715 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa888a3-6d64-4a92-800b-6036dd6e8c8d None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:24.850715 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa888a3-6d64-4a92-800b-6036dd6e8c8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:24.883726 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 968/1929] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:37:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:37:24.894758 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb8c3f9-6ac2-4989-ba21-0b0768c07134 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:24.896368 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-6fb8c3f9-6ac2-4989-ba21-0b0768c07134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:37:24.904054 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-3aa888a3-6d64-4a92-800b-6036dd6e8c8d None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:37:24.909405 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-3aa888a3-6d64-4a92-800b-6036dd6e8c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:37:25.004739 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 962/1930] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:37:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:37:25.007112 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb8c3f9-6ac2-4989-ba21-0b0768c07134 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:25.013223 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea62361-3034-4fb8-af93-14f00b5cb94e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:25.014242 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea62361-3034-4fb8-af93-14f00b5cb94e admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:25.014610 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea62361-3034-4fb8-af93-14f00b5cb94e admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:25.014961 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea62361-3034-4fb8-af93-14f00b5cb94e admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:25.015955 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea62361-3034-4fb8-af93-14f00b5cb94e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:25.018625 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea62361-3034-4fb8-af93-14f00b5cb94e admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:25.027974 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa888a3-6d64-4a92-800b-6036dd6e8c8d None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:25.029809 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-3aa888a3-6d64-4a92-800b-6036dd6e8c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:37:25.119718 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa888a3-6d64-4a92-800b-6036dd6e8c8d None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:25.123218 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aca3f8-da68-47f5-808e-f88873f2ce62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:25.123883 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aca3f8-da68-47f5-808e-f88873f2ce62 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:25.124173 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aca3f8-da68-47f5-808e-f88873f2ce62 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:25.128054 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aca3f8-da68-47f5-808e-f88873f2ce62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:25.128189 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 969/1931] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jul 15 19:37:24 2022] POST /identity/v3/projects => generated 383 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:37:25.129873 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea62361-3034-4fb8-af93-14f00b5cb94e admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:25.137659 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aca3f8-da68-47f5-808e-f88873f2ce62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:25.139578 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aca3f8-da68-47f5-808e-f88873f2ce62 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:25.150536 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 963/1932] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:37:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 19:37:25.157792 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:37:25.203928 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea62361-3034-4fb8-af93-14f00b5cb94e admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:25.212013 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782fef74-d553-4da5-bf91-96e99f354a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:25.213060 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782fef74-d553-4da5-bf91-96e99f354a83 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:25.213175 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782fef74-d553-4da5-bf91-96e99f354a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:25.213420 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782fef74-d553-4da5-bf91-96e99f354a83 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:25.214156 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782fef74-d553-4da5-bf91-96e99f354a83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:25.216323 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782fef74-d553-4da5-bf91-96e99f354a83 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:25.353122 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 970/1933] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jul 15 19:37:25 2022] POST /identity/v3/users => generated 298 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:37:25.357399 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aca3f8-da68-47f5-808e-f88873f2ce62 service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:25.368905 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aca3f8-da68-47f5-808e-f88873f2ce62 service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:25.374893 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9540259-598f-4430-b701-2a0517748a4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:25.375843 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9540259-598f-4430-b701-2a0517748a4a admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:25.376203 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9540259-598f-4430-b701-2a0517748a4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:25.376609 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9540259-598f-4430-b701-2a0517748a4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:25.378485 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9540259-598f-4430-b701-2a0517748a4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:25.380928 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9540259-598f-4430-b701-2a0517748a4a admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:25.393734 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 964/1934] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:37:25 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:37:25.397936 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782fef74-d553-4da5-bf91-96e99f354a83 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:25.409809 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782fef74-d553-4da5-bf91-96e99f354a83 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:25.419143 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c5d1ab-c9a5-4755-ac9f-f735ded0c7fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:25.420061 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c5d1ab-c9a5-4755-ac9f-f735ded0c7fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:25.420342 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c5d1ab-c9a5-4755-ac9f-f735ded0c7fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:25.420670 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c5d1ab-c9a5-4755-ac9f-f735ded0c7fb admin admin] PATH_INFO: `/v3/projects/ade0e02cf91e4d66986c4bb044455da3/users/3e44b29560c34fafb5b5837e5661b9c2/roles/4318f86ba9f74cd4a0bef1c311276657` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:25.436302 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c5d1ab-c9a5-4755-ac9f-f735ded0c7fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ade0e02cf91e4d66986c4bb044455da3, user_id=3e44b29560c34fafb5b5837e5661b9c2, role_id=4318f86ba9f74cd4a0bef1c311276657)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:25.439180 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c5d1ab-c9a5-4755-ac9f-f735ded0c7fb admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:25.563821 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 971/1935] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:37:25 2022] PUT /identity/v3/projects/ade0e02cf91e4d66986c4bb044455da3/users/3e44b29560c34fafb5b5837e5661b9c2/roles/4318f86ba9f74cd4a0bef1c311276657 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:37:25.568775 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9540259-598f-4430-b701-2a0517748a4a admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:25.581707 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9540259-598f-4430-b701-2a0517748a4a admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:25.626647 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:37:25.642441 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5df362-f1fe-4df0-bc03-5ec95993d944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:25.643627 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5df362-f1fe-4df0-bc03-5ec95993d944 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:25.643914 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5df362-f1fe-4df0-bc03-5ec95993d944 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:25.644187 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5df362-f1fe-4df0-bc03-5ec95993d944 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:25.644920 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5df362-f1fe-4df0-bc03-5ec95993d944 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:25.646559 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5df362-f1fe-4df0-bc03-5ec95993d944 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:25.656701 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 965/1936] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:37:25 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 19:37:25.661190 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c5d1ab-c9a5-4755-ac9f-f735ded0c7fb admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:25.671711 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c5d1ab-c9a5-4755-ac9f-f735ded0c7fb admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:25.677994 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c414d8e0-17b1-4326-b826-98315f053900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:25.678922 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c414d8e0-17b1-4326-b826-98315f053900 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:25.679203 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c414d8e0-17b1-4326-b826-98315f053900 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:25.679502 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c414d8e0-17b1-4326-b826-98315f053900 admin admin] PATH_INFO: `/v3/projects/ade0e02cf91e4d66986c4bb044455da3/users/3e44b29560c34fafb5b5837e5661b9c2/roles/4318f86ba9f74cd4a0bef1c311276657` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:25.681633 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c414d8e0-17b1-4326-b826-98315f053900 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ade0e02cf91e4d66986c4bb044455da3, user_id=3e44b29560c34fafb5b5837e5661b9c2, role_id=4318f86ba9f74cd4a0bef1c311276657)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:25.684348 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c414d8e0-17b1-4326-b826-98315f053900 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:25.711486 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 972/1937] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:37:25 2022] PUT /identity/v3/projects/ade0e02cf91e4d66986c4bb044455da3/users/3e44b29560c34fafb5b5837e5661b9c2/roles/4318f86ba9f74cd4a0bef1c311276657 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:37:25.715254 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75972a2d-1ae5-422f-af3f-aff24a5f0da1 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:25.715548 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75972a2d-1ae5-422f-af3f-aff24a5f0da1 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:25.715831 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75972a2d-1ae5-422f-af3f-aff24a5f0da1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:25.743610 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-75972a2d-1ae5-422f-af3f-aff24a5f0da1 None None] MFA Rules not processed for user `3e44b29560c34fafb5b5837e5661b9c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:37:25.745150 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-75972a2d-1ae5-422f-af3f-aff24a5f0da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:37:25.824986 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 966/1938] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:37:25 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:37:25.836444 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c414d8e0-17b1-4326-b826-98315f053900 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:25.838141 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-c414d8e0-17b1-4326-b826-98315f053900 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:37:25.902975 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c414d8e0-17b1-4326-b826-98315f053900 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:25.906629 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ba7873-a530-4cde-8025-b2a3d83f75c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86c2df3d378490ca7ea497a73c19d40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:25.907279 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ba7873-a530-4cde-8025-b2a3d83f75c3 service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:25.907670 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ba7873-a530-4cde-8025-b2a3d83f75c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:25.907998 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ba7873-a530-4cde-8025-b2a3d83f75c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:25.917740 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ba7873-a530-4cde-8025-b2a3d83f75c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:25.919317 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ba7873-a530-4cde-8025-b2a3d83f75c3 service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:25.929509 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 973/1939] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:37:25 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:37:26.000072 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75972a2d-1ae5-422f-af3f-aff24a5f0da1 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:26.001200 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-75972a2d-1ae5-422f-af3f-aff24a5f0da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:37:26.079456 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ba7873-a530-4cde-8025-b2a3d83f75c3 service cinder] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:26.088741 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ba7873-a530-4cde-8025-b2a3d83f75c3 service cinder] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:26.089539 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75972a2d-1ae5-422f-af3f-aff24a5f0da1 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:26.093191 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698b6e97-cd8f-4f20-a9c6-df15d0d7dd57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86c2df3d378490ca7ea497a73c19d40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:26.093627 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b6e97-cd8f-4f20-a9c6-df15d0d7dd57 service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:26.094038 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b6e97-cd8f-4f20-a9c6-df15d0d7dd57 service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:26.094475 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b6e97-cd8f-4f20-a9c6-df15d0d7dd57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:26.096201 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e47244-02d4-47de-bcc9-c5cefb4aa193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:26.097155 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e47244-02d4-47de-bcc9-c5cefb4aa193 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:26.097505 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e47244-02d4-47de-bcc9-c5cefb4aa193 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:26.097703 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e47244-02d4-47de-bcc9-c5cefb4aa193 admin admin] PATH_INFO: `/v3/projects/6fe7daeb43f1454b89181e5d5fbaa351` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:26.098986 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e47244-02d4-47de-bcc9-c5cefb4aa193 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6fe7daeb43f1454b89181e5d5fbaa351)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:26.100980 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e47244-02d4-47de-bcc9-c5cefb4aa193 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:26.167966 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698b6e97-cd8f-4f20-a9c6-df15d0d7dd57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:26.169402 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698b6e97-cd8f-4f20-a9c6-df15d0d7dd57 service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:26.179087 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 974/1940] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:37:26 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 19:37:26.240332 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-94e47244-02d4-47de-bcc9-c5cefb4aa193 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6fe7daeb43f1454b89181e5d5fbaa351', 'request_id': 'req-94e47244-02d4-47de-bcc9-c5cefb4aa193'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:26.322754 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-94e47244-02d4-47de-bcc9-c5cefb4aa193 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6fe7daeb43f1454b89181e5d5fbaa351', 'request_id': 'req-94e47244-02d4-47de-bcc9-c5cefb4aa193'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:26.329053 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-94e47244-02d4-47de-bcc9-c5cefb4aa193 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6fe7daeb43f1454b89181e5d5fbaa351', 'request_id': 'req-94e47244-02d4-47de-bcc9-c5cefb4aa193'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:26.340661 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-94e47244-02d4-47de-bcc9-c5cefb4aa193 admin admin] The token cache is being invalidate because project 6fe7daeb43f1454b89181e5d5fbaa351 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:37:26.340949 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-94e47244-02d4-47de-bcc9-c5cefb4aa193 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:26.341581 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-94e47244-02d4-47de-bcc9-c5cefb4aa193 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:26.343206 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 967/1941] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jul 15 19:37:25 2022] DELETE /identity/v3/projects/6fe7daeb43f1454b89181e5d5fbaa351 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:37:26.350089 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd9edd3-3fec-48c5-81c0-b493484c476c None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:26.350502 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd9edd3-3fec-48c5-81c0-b493484c476c None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:26.350853 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd9edd3-3fec-48c5-81c0-b493484c476c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:26.368660 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-afd9edd3-3fec-48c5-81c0-b493484c476c None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:37:26.369877 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-afd9edd3-3fec-48c5-81c0-b493484c476c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:37:26.379253 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:37:26.432247 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 975/1942] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:37:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:37:26.440257 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a2c7fb-f6c2-4a83-afae-19b5b2ca2562 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:26.440539 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a2c7fb-f6c2-4a83-afae-19b5b2ca2562 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:26.444423 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a2c7fb-f6c2-4a83-afae-19b5b2ca2562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:26.461508 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-95a2c7fb-f6c2-4a83-afae-19b5b2ca2562 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:37:26.462800 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-95a2c7fb-f6c2-4a83-afae-19b5b2ca2562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:37:26.471525 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 968/1943] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:37:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:37:26.475905 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd9edd3-3fec-48c5-81c0-b493484c476c None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:26.486017 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd9edd3-3fec-48c5-81c0-b493484c476c None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:26.490911 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e283a0-11ba-4ca0-83d0-64f3db77afe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:26.491598 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e283a0-11ba-4ca0-83d0-64f3db77afe6 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:26.491884 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e283a0-11ba-4ca0-83d0-64f3db77afe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:26.492161 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e283a0-11ba-4ca0-83d0-64f3db77afe6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:26.492914 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e283a0-11ba-4ca0-83d0-64f3db77afe6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:37:26.493220 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e283a0-11ba-4ca0-83d0-64f3db77afe6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:26.494898 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e283a0-11ba-4ca0-83d0-64f3db77afe6 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:26.506718 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 976/1944] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jul 15 19:37:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:37:26.511225 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a2c7fb-f6c2-4a83-afae-19b5b2ca2562 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:26.521156 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a2c7fb-f6c2-4a83-afae-19b5b2ca2562 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:26.526016 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfaf5d9-5d50-4150-aa16-d9cf67383f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:26.526671 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfaf5d9-5d50-4150-aa16-d9cf67383f9a admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:26.526980 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfaf5d9-5d50-4150-aa16-d9cf67383f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:26.527253 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfaf5d9-5d50-4150-aa16-d9cf67383f9a admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:26.527980 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfaf5d9-5d50-4150-aa16-d9cf67383f9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:26.529908 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfaf5d9-5d50-4150-aa16-d9cf67383f9a admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:26.634580 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 969/1945] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jul 15 19:37:26 2022] POST /identity/v3/projects => generated 375 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:37:26.639120 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e283a0-11ba-4ca0-83d0-64f3db77afe6 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:26.649790 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e283a0-11ba-4ca0-83d0-64f3db77afe6 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:26.705103 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfb91c2-1ce9-4fc5-aaca-7a00d96cd6f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:26.706395 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfb91c2-1ce9-4fc5-aaca-7a00d96cd6f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:26.706667 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfb91c2-1ce9-4fc5-aaca-7a00d96cd6f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:26.706941 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfb91c2-1ce9-4fc5-aaca-7a00d96cd6f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:26.707620 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfb91c2-1ce9-4fc5-aaca-7a00d96cd6f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:26.709629 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfb91c2-1ce9-4fc5-aaca-7a00d96cd6f4 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:26.809387 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 977/1946] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jul 15 19:37:26 2022] POST /identity/v3/users => generated 288 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:37:26.813075 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfaf5d9-5d50-4150-aa16-d9cf67383f9a admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:26.824167 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfaf5d9-5d50-4150-aa16-d9cf67383f9a admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:26.828884 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bcd3d5-e45f-4d72-b9b3-3bc49adee320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:26.829536 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bcd3d5-e45f-4d72-b9b3-3bc49adee320 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:26.829842 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bcd3d5-e45f-4d72-b9b3-3bc49adee320 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:26.830149 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bcd3d5-e45f-4d72-b9b3-3bc49adee320 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:26.831402 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bcd3d5-e45f-4d72-b9b3-3bc49adee320 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:26.832994 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bcd3d5-e45f-4d72-b9b3-3bc49adee320 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:26.843757 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 970/1947] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:37:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:37:26.847644 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfb91c2-1ce9-4fc5-aaca-7a00d96cd6f4 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:26.860660 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfb91c2-1ce9-4fc5-aaca-7a00d96cd6f4 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:26.863946 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c6b51b-a474-4139-a17e-86e9b5eabe84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:26.865221 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c6b51b-a474-4139-a17e-86e9b5eabe84 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:26.865734 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c6b51b-a474-4139-a17e-86e9b5eabe84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:26.866116 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c6b51b-a474-4139-a17e-86e9b5eabe84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:26.866932 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c6b51b-a474-4139-a17e-86e9b5eabe84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:26.868608 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c6b51b-a474-4139-a17e-86e9b5eabe84 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:26.881146 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 978/1948] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:37:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:37:26.886053 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bcd3d5-e45f-4d72-b9b3-3bc49adee320 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:26.897151 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bcd3d5-e45f-4d72-b9b3-3bc49adee320 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:26.906420 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dd6363-f703-41b2-ab11-1ad5bef58f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:26.907309 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd6363-f703-41b2-ab11-1ad5bef58f7f admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:26.907593 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd6363-f703-41b2-ab11-1ad5bef58f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:26.907871 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd6363-f703-41b2-ab11-1ad5bef58f7f admin admin] PATH_INFO: `/v3/projects/2771cec8973344a493780b7f1cf51e18/users/816412dd69984ffa949e2129312b1379/roles/28763c6e70094116b3a4b2895895fa58` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:26.924660 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dd6363-f703-41b2-ab11-1ad5bef58f7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2771cec8973344a493780b7f1cf51e18, user_id=816412dd69984ffa949e2129312b1379, role_id=28763c6e70094116b3a4b2895895fa58)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:26.927453 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dd6363-f703-41b2-ab11-1ad5bef58f7f admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:27.091207 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 971/1949] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:37:26 2022] PUT /identity/v3/projects/2771cec8973344a493780b7f1cf51e18/users/816412dd69984ffa949e2129312b1379/roles/28763c6e70094116b3a4b2895895fa58 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:37:27.094978 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9324c6-1569-4487-9952-da70b9b2be25 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:27.095274 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9324c6-1569-4487-9952-da70b9b2be25 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:27.095561 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9324c6-1569-4487-9952-da70b9b2be25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:27.123927 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-1a9324c6-1569-4487-9952-da70b9b2be25 None None] MFA Rules not processed for user `816412dd69984ffa949e2129312b1379`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:37:27.125157 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-1a9324c6-1569-4487-9952-da70b9b2be25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:37:27.189365 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 979/1950] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:37:27 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:37:27.343974 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:37:27.732135 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dd6363-f703-41b2-ab11-1ad5bef58f7f admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:27.733909 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-00dd6363-f703-41b2-ab11-1ad5bef58f7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:37:27.792406 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dd6363-f703-41b2-ab11-1ad5bef58f7f admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:27.796430 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1bd9f0-bbee-45f7-a787-c47863542d70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86c2df3d378490ca7ea497a73c19d40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:27.797053 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1bd9f0-bbee-45f7-a787-c47863542d70 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:27.797336 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1bd9f0-bbee-45f7-a787-c47863542d70 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:27.797614 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1bd9f0-bbee-45f7-a787-c47863542d70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:27.806660 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1bd9f0-bbee-45f7-a787-c47863542d70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:27.808005 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1bd9f0-bbee-45f7-a787-c47863542d70 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:27.817281 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 972/1951] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:37:27 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:37:34.289983 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469f2d70-b6b8-4e51-adb2-b368e2a8edbf None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:34.291308 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469f2d70-b6b8-4e51-adb2-b368e2a8edbf None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:34.291308 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469f2d70-b6b8-4e51-adb2-b368e2a8edbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:34.309859 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-469f2d70-b6b8-4e51-adb2-b368e2a8edbf None None] MFA Rules not processed for user `816412dd69984ffa949e2129312b1379`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:37:34.311170 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-469f2d70-b6b8-4e51-adb2-b368e2a8edbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:37:34.327259 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 980/1952] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:37:34 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:37:35.504997 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1bd9f0-bbee-45f7-a787-c47863542d70 service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:35.506583 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-6b1bd9f0-bbee-45f7-a787-c47863542d70 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:37:35.589546 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1bd9f0-bbee-45f7-a787-c47863542d70 service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:35.594551 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3569daa8-f305-454c-bc69-ec0a4d1623e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:35.595978 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3569daa8-f305-454c-bc69-ec0a4d1623e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:35.596283 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3569daa8-f305-454c-bc69-ec0a4d1623e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:35.596774 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3569daa8-f305-454c-bc69-ec0a4d1623e0 admin admin] PATH_INFO: `/v3/users/100d42d218574b6ebc2a551e8dd3f482` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:35.598047 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3569daa8-f305-454c-bc69-ec0a4d1623e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=100d42d218574b6ebc2a551e8dd3f482)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:35.600177 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3569daa8-f305-454c-bc69-ec0a4d1623e0 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:35.857592 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:37:35.866039 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3569daa8-f305-454c-bc69-ec0a4d1623e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '100d42d218574b6ebc2a551e8dd3f482'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:35.877732 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3569daa8-f305-454c-bc69-ec0a4d1623e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '100d42d218574b6ebc2a551e8dd3f482'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:35.950836 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3569daa8-f305-454c-bc69-ec0a4d1623e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '100d42d218574b6ebc2a551e8dd3f482'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:35.951437 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3569daa8-f305-454c-bc69-ec0a4d1623e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '100d42d218574b6ebc2a551e8dd3f482'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:35.959338 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3569daa8-f305-454c-bc69-ec0a4d1623e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '100d42d218574b6ebc2a551e8dd3f482'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:35.965538 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 973/1953] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jul 15 19:37:35 2022] DELETE /identity/v3/users/100d42d218574b6ebc2a551e8dd3f482 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:37:35.970001 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62d481d-4df7-4975-9e2a-654aa01899d5 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:35.971036 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62d481d-4df7-4975-9e2a-654aa01899d5 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:35.976684 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62d481d-4df7-4975-9e2a-654aa01899d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:35.977919 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:37:35.988531 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-f62d481d-4df7-4975-9e2a-654aa01899d5 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:37:35.989734 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-f62d481d-4df7-4975-9e2a-654aa01899d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:37:36.059801 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 981/1954] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:37:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:37:36.070071 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3569daa8-f305-454c-bc69-ec0a4d1623e0 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:36.071717 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-3569daa8-f305-454c-bc69-ec0a4d1623e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:37:36.132596 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3569daa8-f305-454c-bc69-ec0a4d1623e0 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:36.136598 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0585da-e2d0-4971-91cb-6111788956e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:36.137256 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0585da-e2d0-4971-91cb-6111788956e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:36.137548 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0585da-e2d0-4971-91cb-6111788956e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:36.137866 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0585da-e2d0-4971-91cb-6111788956e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:36.153271 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0585da-e2d0-4971-91cb-6111788956e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:36.155376 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0585da-e2d0-4971-91cb-6111788956e4 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:36.166680 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 974/1955] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:37:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:37:36.704783 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62d481d-4df7-4975-9e2a-654aa01899d5 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:36.706488 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-f62d481d-4df7-4975-9e2a-654aa01899d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:37:36.720090 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62d481d-4df7-4975-9e2a-654aa01899d5 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:36.725999 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9b0476-8ee4-4a1b-942f-b3ed75bbae81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:36.727069 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9b0476-8ee4-4a1b-942f-b3ed75bbae81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:36.727458 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9b0476-8ee4-4a1b-942f-b3ed75bbae81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:36.727831 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9b0476-8ee4-4a1b-942f-b3ed75bbae81 admin admin] PATH_INFO: `/v3/projects/392cba1373bf42a39dc647c3421031c0` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:36.729220 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9b0476-8ee4-4a1b-942f-b3ed75bbae81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=392cba1373bf42a39dc647c3421031c0)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:36.731714 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9b0476-8ee4-4a1b-942f-b3ed75bbae81 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:36.855098 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-1c9b0476-8ee4-4a1b-942f-b3ed75bbae81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '392cba1373bf42a39dc647c3421031c0', 'request_id': 'req-1c9b0476-8ee4-4a1b-942f-b3ed75bbae81'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:36.948403 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-1c9b0476-8ee4-4a1b-942f-b3ed75bbae81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '392cba1373bf42a39dc647c3421031c0', 'request_id': 'req-1c9b0476-8ee4-4a1b-942f-b3ed75bbae81'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:36.955341 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-1c9b0476-8ee4-4a1b-942f-b3ed75bbae81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '392cba1373bf42a39dc647c3421031c0', 'request_id': 'req-1c9b0476-8ee4-4a1b-942f-b3ed75bbae81'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:36.966540 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-1c9b0476-8ee4-4a1b-942f-b3ed75bbae81 admin admin] The token cache is being invalidate because project 392cba1373bf42a39dc647c3421031c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:37:36.966927 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-1c9b0476-8ee4-4a1b-942f-b3ed75bbae81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:36.967677 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-1c9b0476-8ee4-4a1b-942f-b3ed75bbae81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:36.969417 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 982/1956] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jul 15 19:37:36 2022] DELETE /identity/v3/projects/392cba1373bf42a39dc647c3421031c0 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:37:36.973917 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0585da-e2d0-4971-91cb-6111788956e4 service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:36.983838 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0585da-e2d0-4971-91cb-6111788956e4 service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:37.051392 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b11d854-b714-4295-8b4d-3fae6dfd8fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:37.052265 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b11d854-b714-4295-8b4d-3fae6dfd8fc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:37.052542 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b11d854-b714-4295-8b4d-3fae6dfd8fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:37.052814 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b11d854-b714-4295-8b4d-3fae6dfd8fc7 admin admin] PATH_INFO: `/v3/users/3e44b29560c34fafb5b5837e5661b9c2` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:37.054024 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b11d854-b714-4295-8b4d-3fae6dfd8fc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e44b29560c34fafb5b5837e5661b9c2)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:37.055947 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b11d854-b714-4295-8b4d-3fae6dfd8fc7 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:37.238965 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4b11d854-b714-4295-8b4d-3fae6dfd8fc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e44b29560c34fafb5b5837e5661b9c2'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:37.246753 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:37:37.251153 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4b11d854-b714-4295-8b4d-3fae6dfd8fc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e44b29560c34fafb5b5837e5661b9c2'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:37.362299 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4b11d854-b714-4295-8b4d-3fae6dfd8fc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e44b29560c34fafb5b5837e5661b9c2'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:37.362868 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4b11d854-b714-4295-8b4d-3fae6dfd8fc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e44b29560c34fafb5b5837e5661b9c2'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:37.370163 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4b11d854-b714-4295-8b4d-3fae6dfd8fc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e44b29560c34fafb5b5837e5661b9c2'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:37.376049 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 975/1957] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jul 15 19:37:36 2022] DELETE /identity/v3/users/3e44b29560c34fafb5b5837e5661b9c2 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:37:37.944495 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9b0476-8ee4-4a1b-942f-b3ed75bbae81 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:37.945978 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-1c9b0476-8ee4-4a1b-942f-b3ed75bbae81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:37:37.959332 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9b0476-8ee4-4a1b-942f-b3ed75bbae81 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:38.018121 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a646b4b-2098-4309-93cd-794d6e92a390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:38.019017 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a646b4b-2098-4309-93cd-794d6e92a390 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:38.019296 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a646b4b-2098-4309-93cd-794d6e92a390 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:38.019571 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a646b4b-2098-4309-93cd-794d6e92a390 admin admin] PATH_INFO: `/v3/projects/ade0e02cf91e4d66986c4bb044455da3` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:38.020753 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a646b4b-2098-4309-93cd-794d6e92a390 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ade0e02cf91e4d66986c4bb044455da3)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:38.022716 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a646b4b-2098-4309-93cd-794d6e92a390 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:38.127782 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:37:38.140982 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-2a646b4b-2098-4309-93cd-794d6e92a390 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ade0e02cf91e4d66986c4bb044455da3', 'request_id': 'req-2a646b4b-2098-4309-93cd-794d6e92a390'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:38.242432 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-2a646b4b-2098-4309-93cd-794d6e92a390 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ade0e02cf91e4d66986c4bb044455da3', 'request_id': 'req-2a646b4b-2098-4309-93cd-794d6e92a390'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:38.249367 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-2a646b4b-2098-4309-93cd-794d6e92a390 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ade0e02cf91e4d66986c4bb044455da3', 'request_id': 'req-2a646b4b-2098-4309-93cd-794d6e92a390'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:38.260471 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-2a646b4b-2098-4309-93cd-794d6e92a390 admin admin] The token cache is being invalidate because project ade0e02cf91e4d66986c4bb044455da3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:37:38.260741 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-2a646b4b-2098-4309-93cd-794d6e92a390 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:38.261378 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-2a646b4b-2098-4309-93cd-794d6e92a390 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:38.262948 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 983/1958] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jul 15 19:37:37 2022] DELETE /identity/v3/projects/ade0e02cf91e4d66986c4bb044455da3 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:37:38.272124 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f942e9c1-fe35-4f66-a1df-b975fcca9e45 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:38.272516 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f942e9c1-fe35-4f66-a1df-b975fcca9e45 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:38.272687 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f942e9c1-fe35-4f66-a1df-b975fcca9e45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:38.290993 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-f942e9c1-fe35-4f66-a1df-b975fcca9e45 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:37:38.292220 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-f942e9c1-fe35-4f66-a1df-b975fcca9e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:37:38.351465 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 976/1959] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:37:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:37:38.359784 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e3dde1-6f77-4149-83c3-1b83a52961a6 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:38.360073 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e3dde1-6f77-4149-83c3-1b83a52961a6 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:38.360355 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e3dde1-6f77-4149-83c3-1b83a52961a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:38.378899 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-b6e3dde1-6f77-4149-83c3-1b83a52961a6 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:37:38.386795 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-b6e3dde1-6f77-4149-83c3-1b83a52961a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:37:38.388424 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 984/1960] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:37:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:37:38.393519 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f942e9c1-fe35-4f66-a1df-b975fcca9e45 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:38.403420 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f942e9c1-fe35-4f66-a1df-b975fcca9e45 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:38.408246 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf163df-4683-448e-9c1d-b3d73217ffb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:38.408903 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf163df-4683-448e-9c1d-b3d73217ffb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:38.409188 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf163df-4683-448e-9c1d-b3d73217ffb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:38.409467 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf163df-4683-448e-9c1d-b3d73217ffb3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:38.410237 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf163df-4683-448e-9c1d-b3d73217ffb3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:37:38.410548 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf163df-4683-448e-9c1d-b3d73217ffb3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:38.412135 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf163df-4683-448e-9c1d-b3d73217ffb3 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:38.423854 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 977/1961] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jul 15 19:37:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:37:38.428207 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e3dde1-6f77-4149-83c3-1b83a52961a6 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:38.438049 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e3dde1-6f77-4149-83c3-1b83a52961a6 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:38.443498 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a508377-e706-4d49-af4b-8fde9e6f90b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:38.444153 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a508377-e706-4d49-af4b-8fde9e6f90b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:38.444439 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a508377-e706-4d49-af4b-8fde9e6f90b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:38.444717 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a508377-e706-4d49-af4b-8fde9e6f90b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:38.445422 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a508377-e706-4d49-af4b-8fde9e6f90b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:38.447612 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a508377-e706-4d49-af4b-8fde9e6f90b6 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:38.546598 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 985/1962] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jul 15 19:37:38 2022] POST /identity/v3/projects => generated 379 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:37:38.551140 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf163df-4683-448e-9c1d-b3d73217ffb3 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:38.561678 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf163df-4683-448e-9c1d-b3d73217ffb3 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:38.619807 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05eb0d3-f2f6-4ba9-bd30-0d40558fe828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:38.620484 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05eb0d3-f2f6-4ba9-bd30-0d40558fe828 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:38.620774 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05eb0d3-f2f6-4ba9-bd30-0d40558fe828 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:38.621053 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05eb0d3-f2f6-4ba9-bd30-0d40558fe828 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:38.621745 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05eb0d3-f2f6-4ba9-bd30-0d40558fe828 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:38.623773 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05eb0d3-f2f6-4ba9-bd30-0d40558fe828 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:38.801419 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 978/1963] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jul 15 19:37:38 2022] POST /identity/v3/users => generated 290 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:37:38.808454 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a508377-e706-4d49-af4b-8fde9e6f90b6 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:38.820133 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a508377-e706-4d49-af4b-8fde9e6f90b6 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:38.824769 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ebc21e-0532-4531-b433-753f001cc691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:38.825429 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ebc21e-0532-4531-b433-753f001cc691 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:38.825711 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ebc21e-0532-4531-b433-753f001cc691 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:38.826031 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ebc21e-0532-4531-b433-753f001cc691 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:38.826763 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ebc21e-0532-4531-b433-753f001cc691 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:38.828518 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ebc21e-0532-4531-b433-753f001cc691 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:38.841968 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 986/1964] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:37:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:37:38.846574 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05eb0d3-f2f6-4ba9-bd30-0d40558fe828 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:38.856597 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05eb0d3-f2f6-4ba9-bd30-0d40558fe828 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:38.862120 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579b14b9-9c10-4548-ba22-9c0c11d5b29f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:38.862787 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b14b9-9c10-4548-ba22-9c0c11d5b29f admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:38.863070 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b14b9-9c10-4548-ba22-9c0c11d5b29f admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:38.863348 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b14b9-9c10-4548-ba22-9c0c11d5b29f admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:38.864079 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579b14b9-9c10-4548-ba22-9c0c11d5b29f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:38.865704 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579b14b9-9c10-4548-ba22-9c0c11d5b29f admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:38.875773 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 979/1965] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:37:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:37:38.880374 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ebc21e-0532-4531-b433-753f001cc691 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:38.890468 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ebc21e-0532-4531-b433-753f001cc691 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:38.894969 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdf34d0-f3d9-44a4-be60-30ae3c5bbfe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:38.895843 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdf34d0-f3d9-44a4-be60-30ae3c5bbfe9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:38.896127 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdf34d0-f3d9-44a4-be60-30ae3c5bbfe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:38.896405 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdf34d0-f3d9-44a4-be60-30ae3c5bbfe9 admin admin] PATH_INFO: `/v3/projects/905e399c9b6c4b61a626332780306517/users/bf8ba59c97f44aabb5cd5e2fb13a71c3/roles/28763c6e70094116b3a4b2895895fa58` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:38.911314 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdf34d0-f3d9-44a4-be60-30ae3c5bbfe9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=905e399c9b6c4b61a626332780306517, user_id=bf8ba59c97f44aabb5cd5e2fb13a71c3, role_id=28763c6e70094116b3a4b2895895fa58)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:38.914071 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdf34d0-f3d9-44a4-be60-30ae3c5bbfe9 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:39.002892 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 987/1966] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:37:38 2022] PUT /identity/v3/projects/905e399c9b6c4b61a626332780306517/users/bf8ba59c97f44aabb5cd5e2fb13a71c3/roles/28763c6e70094116b3a4b2895895fa58 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:37:39.014550 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57508ae-b812-4faf-a5fb-1b1cca0f5cd4 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:39.014839 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57508ae-b812-4faf-a5fb-1b1cca0f5cd4 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:39.015118 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57508ae-b812-4faf-a5fb-1b1cca0f5cd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:39.045910 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-d57508ae-b812-4faf-a5fb-1b1cca0f5cd4 None None] MFA Rules not processed for user `bf8ba59c97f44aabb5cd5e2fb13a71c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:37:39.047147 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d57508ae-b812-4faf-a5fb-1b1cca0f5cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:37:39.076871 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:37:39.110482 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 980/1967] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:37:39 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:37:39.115624 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdf34d0-f3d9-44a4-be60-30ae3c5bbfe9 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:39.178144 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdf34d0-f3d9-44a4-be60-30ae3c5bbfe9 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:39.185926 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7184bb6-5dd0-43d1-8a87-b56b4811e45a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:39.185926 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7184bb6-5dd0-43d1-8a87-b56b4811e45a admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:39.185926 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7184bb6-5dd0-43d1-8a87-b56b4811e45a admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:39.185926 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7184bb6-5dd0-43d1-8a87-b56b4811e45a admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:39.186302 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7184bb6-5dd0-43d1-8a87-b56b4811e45a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:39.186486 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7184bb6-5dd0-43d1-8a87-b56b4811e45a admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:39.309357 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 988/1968] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jul 15 19:37:39 2022] POST /identity/v3/projects => generated 379 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:37:39.312773 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57508ae-b812-4faf-a5fb-1b1cca0f5cd4 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:39.323676 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57508ae-b812-4faf-a5fb-1b1cca0f5cd4 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:39.381276 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b16ea5a-cf3d-44a8-a299-5edbe38fdb79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:39.381985 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b16ea5a-cf3d-44a8-a299-5edbe38fdb79 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:39.382316 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b16ea5a-cf3d-44a8-a299-5edbe38fdb79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:39.386390 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b16ea5a-cf3d-44a8-a299-5edbe38fdb79 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:39.386390 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b16ea5a-cf3d-44a8-a299-5edbe38fdb79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:39.386390 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b16ea5a-cf3d-44a8-a299-5edbe38fdb79 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:39.554944 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 981/1969] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jul 15 19:37:39 2022] POST /identity/v3/users => generated 290 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:37:39.558964 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7184bb6-5dd0-43d1-8a87-b56b4811e45a admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:39.569225 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7184bb6-5dd0-43d1-8a87-b56b4811e45a admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:39.573839 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6d94e6-aac8-43d9-89e1-2a755e930456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:39.574494 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6d94e6-aac8-43d9-89e1-2a755e930456 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:39.574775 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6d94e6-aac8-43d9-89e1-2a755e930456 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:39.575048 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6d94e6-aac8-43d9-89e1-2a755e930456 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:39.575765 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6d94e6-aac8-43d9-89e1-2a755e930456 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:39.577339 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6d94e6-aac8-43d9-89e1-2a755e930456 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:39.588410 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 989/1970] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:37:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:37:39.592326 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b16ea5a-cf3d-44a8-a299-5edbe38fdb79 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:39.603173 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b16ea5a-cf3d-44a8-a299-5edbe38fdb79 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:39.607971 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b0616b-fcd9-4c50-8f5a-82bd71642414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:39.608653 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b0616b-fcd9-4c50-8f5a-82bd71642414 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:39.608934 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b0616b-fcd9-4c50-8f5a-82bd71642414 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:39.609208 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b0616b-fcd9-4c50-8f5a-82bd71642414 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:39.609971 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b0616b-fcd9-4c50-8f5a-82bd71642414 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:39.611638 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b0616b-fcd9-4c50-8f5a-82bd71642414 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:39.621680 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 982/1971] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:37:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:37:39.625552 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6d94e6-aac8-43d9-89e1-2a755e930456 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:39.635468 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6d94e6-aac8-43d9-89e1-2a755e930456 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:39.639896 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a177f2ea-799a-431e-aaa1-1c603ed356ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:39.640847 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a177f2ea-799a-431e-aaa1-1c603ed356ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:39.641170 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a177f2ea-799a-431e-aaa1-1c603ed356ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:39.641395 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a177f2ea-799a-431e-aaa1-1c603ed356ad admin admin] PATH_INFO: `/v3/projects/ca6edeb3588a48a39e50017c90dfd4de/users/7b849dfbe38c4825a1f12cca102cfde9/roles/28763c6e70094116b3a4b2895895fa58` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:39.656812 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a177f2ea-799a-431e-aaa1-1c603ed356ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca6edeb3588a48a39e50017c90dfd4de, user_id=7b849dfbe38c4825a1f12cca102cfde9, role_id=28763c6e70094116b3a4b2895895fa58)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:39.661173 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a177f2ea-799a-431e-aaa1-1c603ed356ad admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:39.737609 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 990/1972] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:37:39 2022] PUT /identity/v3/projects/ca6edeb3588a48a39e50017c90dfd4de/users/7b849dfbe38c4825a1f12cca102cfde9/roles/28763c6e70094116b3a4b2895895fa58 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:37:39.747954 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd06f0d7-71c8-4647-a21b-53b002b17707 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:39.748223 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd06f0d7-71c8-4647-a21b-53b002b17707 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:39.748501 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd06f0d7-71c8-4647-a21b-53b002b17707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:39.777690 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-cd06f0d7-71c8-4647-a21b-53b002b17707 None None] MFA Rules not processed for user `7b849dfbe38c4825a1f12cca102cfde9`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:37:39.778941 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-cd06f0d7-71c8-4647-a21b-53b002b17707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:37:39.867533 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 983/1973] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:37:39 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:37:39.872823 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a177f2ea-799a-431e-aaa1-1c603ed356ad admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:39.879352 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:37:39.935661 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a177f2ea-799a-431e-aaa1-1c603ed356ad admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:39.941324 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8351e94d-f641-417b-a271-6c1f1c63e1db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:39.942018 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8351e94d-f641-417b-a271-6c1f1c63e1db admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:39.942301 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8351e94d-f641-417b-a271-6c1f1c63e1db admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:39.942576 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8351e94d-f641-417b-a271-6c1f1c63e1db admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:39.943278 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8351e94d-f641-417b-a271-6c1f1c63e1db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:39.945179 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8351e94d-f641-417b-a271-6c1f1c63e1db admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:40.032445 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 991/1974] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jul 15 19:37:39 2022] POST /identity/v3/projects => generated 377 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:37:40.036568 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd06f0d7-71c8-4647-a21b-53b002b17707 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:40.047607 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd06f0d7-71c8-4647-a21b-53b002b17707 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:40.085961 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:37:40.103747 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503c332a-c733-4f4d-a2b2-21ece6c4bc1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:40.104423 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c332a-c733-4f4d-a2b2-21ece6c4bc1a admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:40.104708 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c332a-c733-4f4d-a2b2-21ece6c4bc1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:40.104987 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c332a-c733-4f4d-a2b2-21ece6c4bc1a admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:40.105676 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503c332a-c733-4f4d-a2b2-21ece6c4bc1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:40.107647 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503c332a-c733-4f4d-a2b2-21ece6c4bc1a admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:40.200907 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 984/1975] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jul 15 19:37:40 2022] POST /identity/v3/users => generated 295 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:37:40.205137 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8351e94d-f641-417b-a271-6c1f1c63e1db admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:40.215588 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8351e94d-f641-417b-a271-6c1f1c63e1db admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:40.220526 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51486baa-9582-4fe2-83b7-3d08a69ff67e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:40.221196 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51486baa-9582-4fe2-83b7-3d08a69ff67e admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:40.221483 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51486baa-9582-4fe2-83b7-3d08a69ff67e admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:40.221788 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51486baa-9582-4fe2-83b7-3d08a69ff67e admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:40.222519 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51486baa-9582-4fe2-83b7-3d08a69ff67e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:40.224188 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51486baa-9582-4fe2-83b7-3d08a69ff67e admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:40.235109 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 992/1976] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:37:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:37:40.239657 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503c332a-c733-4f4d-a2b2-21ece6c4bc1a admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:40.250020 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503c332a-c733-4f4d-a2b2-21ece6c4bc1a admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:40.255329 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd71c36-62d5-40ee-8661-334c23f77fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:40.256210 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd71c36-62d5-40ee-8661-334c23f77fe7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:40.256490 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd71c36-62d5-40ee-8661-334c23f77fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:40.256763 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd71c36-62d5-40ee-8661-334c23f77fe7 admin admin] PATH_INFO: `/v3/projects/deff5edeb3e148208182b715fd5015c6/users/ed118dd44504404f880b91541e23ffac/roles/4318f86ba9f74cd4a0bef1c311276657` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:40.271945 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd71c36-62d5-40ee-8661-334c23f77fe7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=deff5edeb3e148208182b715fd5015c6, user_id=ed118dd44504404f880b91541e23ffac, role_id=4318f86ba9f74cd4a0bef1c311276657)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:40.274643 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd71c36-62d5-40ee-8661-334c23f77fe7 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:40.355064 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 985/1977] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:37:40 2022] PUT /identity/v3/projects/deff5edeb3e148208182b715fd5015c6/users/ed118dd44504404f880b91541e23ffac/roles/4318f86ba9f74cd4a0bef1c311276657 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:37:40.367305 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51486baa-9582-4fe2-83b7-3d08a69ff67e admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:40.369726 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51486baa-9582-4fe2-83b7-3d08a69ff67e admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:40.442457 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8985949-8679-4c18-bb40-69dfacfbf6e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:40.442696 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8985949-8679-4c18-bb40-69dfacfbf6e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:40.442961 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8985949-8679-4c18-bb40-69dfacfbf6e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:40.443247 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8985949-8679-4c18-bb40-69dfacfbf6e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:40.443995 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8985949-8679-4c18-bb40-69dfacfbf6e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:40.445631 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8985949-8679-4c18-bb40-69dfacfbf6e8 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:40.459968 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 993/1978] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:37:40 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:37:40.461334 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd71c36-62d5-40ee-8661-334c23f77fe7 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:40.472331 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd71c36-62d5-40ee-8661-334c23f77fe7 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:40.484067 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a490ad07-2f9d-45fc-adff-f97cf50ebb99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:40.484976 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a490ad07-2f9d-45fc-adff-f97cf50ebb99 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:40.485257 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a490ad07-2f9d-45fc-adff-f97cf50ebb99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:40.485528 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a490ad07-2f9d-45fc-adff-f97cf50ebb99 admin admin] PATH_INFO: `/v3/projects/deff5edeb3e148208182b715fd5015c6/users/ed118dd44504404f880b91541e23ffac/roles/4318f86ba9f74cd4a0bef1c311276657` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:40.488068 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a490ad07-2f9d-45fc-adff-f97cf50ebb99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=deff5edeb3e148208182b715fd5015c6, user_id=ed118dd44504404f880b91541e23ffac, role_id=4318f86ba9f74cd4a0bef1c311276657)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:40.490811 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a490ad07-2f9d-45fc-adff-f97cf50ebb99 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:40.501321 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 986/1979] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:37:40 2022] PUT /identity/v3/projects/deff5edeb3e148208182b715fd5015c6/users/ed118dd44504404f880b91541e23ffac/roles/4318f86ba9f74cd4a0bef1c311276657 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:37:40.512354 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218333ef-a418-4867-ab04-e419263b62df None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:40.512644 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218333ef-a418-4867-ab04-e419263b62df None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:40.512920 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218333ef-a418-4867-ab04-e419263b62df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:40.540665 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-218333ef-a418-4867-ab04-e419263b62df None None] MFA Rules not processed for user `ed118dd44504404f880b91541e23ffac`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:37:40.541932 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-218333ef-a418-4867-ab04-e419263b62df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:37:40.615007 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:37:40.618618 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 994/1980] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:37:40 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:37:40.630015 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a490ad07-2f9d-45fc-adff-f97cf50ebb99 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:40.631786 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-a490ad07-2f9d-45fc-adff-f97cf50ebb99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:37:40.690848 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a490ad07-2f9d-45fc-adff-f97cf50ebb99 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:40.694664 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b3e24a-4a3b-4976-8975-8f26dc9b8578 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86c2df3d378490ca7ea497a73c19d40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:40.695347 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b3e24a-4a3b-4976-8975-8f26dc9b8578 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:40.695633 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b3e24a-4a3b-4976-8975-8f26dc9b8578 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:40.695904 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b3e24a-4a3b-4976-8975-8f26dc9b8578 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:40.705351 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b3e24a-4a3b-4976-8975-8f26dc9b8578 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:40.706822 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b3e24a-4a3b-4976-8975-8f26dc9b8578 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:40.716640 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 987/1981] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:37:40 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:37:40.804866 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218333ef-a418-4867-ab04-e419263b62df None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:40.806320 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-218333ef-a418-4867-ab04-e419263b62df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:37:40.818195 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218333ef-a418-4867-ab04-e419263b62df None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:40.821729 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1267bd41-5f3f-42c9-8c93-e230caafa5ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86c2df3d378490ca7ea497a73c19d40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:40.822404 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1267bd41-5f3f-42c9-8c93-e230caafa5ab service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:40.822689 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1267bd41-5f3f-42c9-8c93-e230caafa5ab service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:40.822960 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1267bd41-5f3f-42c9-8c93-e230caafa5ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:40.880260 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1267bd41-5f3f-42c9-8c93-e230caafa5ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:40.881683 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1267bd41-5f3f-42c9-8c93-e230caafa5ab service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:40.891039 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 995/1982] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:37:40 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:37:44.750900 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b3e24a-4a3b-4976-8975-8f26dc9b8578 service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:44.755094 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-12b3e24a-4a3b-4976-8975-8f26dc9b8578 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:37:44.832667 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b3e24a-4a3b-4976-8975-8f26dc9b8578 service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:44.837396 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58385e4f-bbc5-4ea2-ad39-b39b905c0426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:44.838419 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58385e4f-bbc5-4ea2-ad39-b39b905c0426 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:44.838768 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58385e4f-bbc5-4ea2-ad39-b39b905c0426 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:44.839117 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58385e4f-bbc5-4ea2-ad39-b39b905c0426 admin admin] PATH_INFO: `/v3/users/e7ac8a635f044826b308b0b4d26e65c2` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:44.840443 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58385e4f-bbc5-4ea2-ad39-b39b905c0426 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7ac8a635f044826b308b0b4d26e65c2)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:44.842509 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58385e4f-bbc5-4ea2-ad39-b39b905c0426 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:45.063486 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:37:45.076005 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-58385e4f-bbc5-4ea2-ad39-b39b905c0426 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7ac8a635f044826b308b0b4d26e65c2'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:45.088251 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-58385e4f-bbc5-4ea2-ad39-b39b905c0426 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7ac8a635f044826b308b0b4d26e65c2'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:45.196668 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-58385e4f-bbc5-4ea2-ad39-b39b905c0426 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7ac8a635f044826b308b0b4d26e65c2'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:45.197293 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-58385e4f-bbc5-4ea2-ad39-b39b905c0426 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7ac8a635f044826b308b0b4d26e65c2'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:45.204619 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-58385e4f-bbc5-4ea2-ad39-b39b905c0426 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7ac8a635f044826b308b0b4d26e65c2'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:45.210543 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 988/1983] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jul 15 19:37:44 2022] DELETE /identity/v3/users/e7ac8a635f044826b308b0b4d26e65c2 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:37:45.215157 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb712b6f-d891-44b6-b99a-f8e13798684a None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:45.215639 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb712b6f-d891-44b6-b99a-f8e13798684a None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:45.215766 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb712b6f-d891-44b6-b99a-f8e13798684a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:45.232925 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-eb712b6f-d891-44b6-b99a-f8e13798684a None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:37:45.234163 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-eb712b6f-d891-44b6-b99a-f8e13798684a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:37:45.292014 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 996/1984] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:37:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:37:45.302973 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58385e4f-bbc5-4ea2-ad39-b39b905c0426 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:45.304447 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-58385e4f-bbc5-4ea2-ad39-b39b905c0426 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:37:45.363142 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58385e4f-bbc5-4ea2-ad39-b39b905c0426 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:45.366712 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6469d73-f11b-4bfd-8e63-c753f504b457 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:45.367352 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6469d73-f11b-4bfd-8e63-c753f504b457 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:45.367630 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6469d73-f11b-4bfd-8e63-c753f504b457 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:45.367902 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6469d73-f11b-4bfd-8e63-c753f504b457 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:45.376950 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6469d73-f11b-4bfd-8e63-c753f504b457 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:45.378551 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6469d73-f11b-4bfd-8e63-c753f504b457 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:45.390162 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 989/1985] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:37:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:37:45.826933 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb712b6f-d891-44b6-b99a-f8e13798684a None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:45.828444 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-eb712b6f-d891-44b6-b99a-f8e13798684a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:37:45.840624 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb712b6f-d891-44b6-b99a-f8e13798684a None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:45.845176 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a66885e-9889-4028-bb71-1882682d3522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:45.846149 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a66885e-9889-4028-bb71-1882682d3522 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:45.846439 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a66885e-9889-4028-bb71-1882682d3522 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:45.846714 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a66885e-9889-4028-bb71-1882682d3522 admin admin] PATH_INFO: `/v3/projects/92d449cf5d5d4c10a2d50bfe740312cb` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:45.848524 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a66885e-9889-4028-bb71-1882682d3522 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92d449cf5d5d4c10a2d50bfe740312cb)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:45.850774 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a66885e-9889-4028-bb71-1882682d3522 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:46.014885 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-3a66885e-9889-4028-bb71-1882682d3522 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92d449cf5d5d4c10a2d50bfe740312cb', 'request_id': 'req-3a66885e-9889-4028-bb71-1882682d3522'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:46.114380 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-3a66885e-9889-4028-bb71-1882682d3522 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92d449cf5d5d4c10a2d50bfe740312cb', 'request_id': 'req-3a66885e-9889-4028-bb71-1882682d3522'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:46.121100 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-3a66885e-9889-4028-bb71-1882682d3522 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92d449cf5d5d4c10a2d50bfe740312cb', 'request_id': 'req-3a66885e-9889-4028-bb71-1882682d3522'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:46.133383 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-3a66885e-9889-4028-bb71-1882682d3522 admin admin] The token cache is being invalidate because project 92d449cf5d5d4c10a2d50bfe740312cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:37:46.133676 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-3a66885e-9889-4028-bb71-1882682d3522 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:46.134676 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-3a66885e-9889-4028-bb71-1882682d3522 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:46.136249 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 997/1986] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jul 15 19:37:45 2022] DELETE /identity/v3/projects/92d449cf5d5d4c10a2d50bfe740312cb => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:37:53.854646 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6469d73-f11b-4bfd-8e63-c753f504b457 service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:53.856149 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b6469d73-f11b-4bfd-8e63-c753f504b457 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:37:53.869740 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6469d73-f11b-4bfd-8e63-c753f504b457 service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:53.925651 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36e29fc-0b7a-4da8-b8f6-473acbd09ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:53.926555 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e29fc-0b7a-4da8-b8f6-473acbd09ba2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:53.926830 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e29fc-0b7a-4da8-b8f6-473acbd09ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:53.927097 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e29fc-0b7a-4da8-b8f6-473acbd09ba2 admin admin] PATH_INFO: `/v3/users/bf8ba59c97f44aabb5cd5e2fb13a71c3` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:53.928284 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36e29fc-0b7a-4da8-b8f6-473acbd09ba2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf8ba59c97f44aabb5cd5e2fb13a71c3)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:53.930243 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36e29fc-0b7a-4da8-b8f6-473acbd09ba2 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:54.159181 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b36e29fc-0b7a-4da8-b8f6-473acbd09ba2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf8ba59c97f44aabb5cd5e2fb13a71c3'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:54.167370 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:37:54.171680 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b36e29fc-0b7a-4da8-b8f6-473acbd09ba2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf8ba59c97f44aabb5cd5e2fb13a71c3'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:54.274400 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b36e29fc-0b7a-4da8-b8f6-473acbd09ba2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf8ba59c97f44aabb5cd5e2fb13a71c3'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:54.275019 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b36e29fc-0b7a-4da8-b8f6-473acbd09ba2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf8ba59c97f44aabb5cd5e2fb13a71c3'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:54.282780 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b36e29fc-0b7a-4da8-b8f6-473acbd09ba2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf8ba59c97f44aabb5cd5e2fb13a71c3'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:54.289880 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 990/1987] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jul 15 19:37:53 2022] DELETE /identity/v3/users/bf8ba59c97f44aabb5cd5e2fb13a71c3 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:37:54.294880 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c683729-1e86-4b6e-b0b5-af72c3d7c28d None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:54.295362 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c683729-1e86-4b6e-b0b5-af72c3d7c28d None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:54.295569 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c683729-1e86-4b6e-b0b5-af72c3d7c28d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:54.316833 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-7c683729-1e86-4b6e-b0b5-af72c3d7c28d None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:37:54.318097 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-7c683729-1e86-4b6e-b0b5-af72c3d7c28d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:37:54.336069 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:37:54.379669 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 998/1988] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:37:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:37:54.389744 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36e29fc-0b7a-4da8-b8f6-473acbd09ba2 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:54.391240 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b36e29fc-0b7a-4da8-b8f6-473acbd09ba2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:37:54.474682 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36e29fc-0b7a-4da8-b8f6-473acbd09ba2 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:54.478503 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c9edda-8ac2-47c2-a0c0-6138973441d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:54.482298 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c9edda-8ac2-47c2-a0c0-6138973441d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:54.482298 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c9edda-8ac2-47c2-a0c0-6138973441d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:54.482298 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c9edda-8ac2-47c2-a0c0-6138973441d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:54.490529 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c9edda-8ac2-47c2-a0c0-6138973441d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:54.493274 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c9edda-8ac2-47c2-a0c0-6138973441d8 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:54.506905 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 991/1989] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:37:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:37:55.067216 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c683729-1e86-4b6e-b0b5-af72c3d7c28d None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:55.068748 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-7c683729-1e86-4b6e-b0b5-af72c3d7c28d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:37:55.085318 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c683729-1e86-4b6e-b0b5-af72c3d7c28d None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:55.092554 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed55ff9-0f63-47a4-be3b-de8144cb2393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:55.093447 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed55ff9-0f63-47a4-be3b-de8144cb2393 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:55.093757 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed55ff9-0f63-47a4-be3b-de8144cb2393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:55.094217 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed55ff9-0f63-47a4-be3b-de8144cb2393 admin admin] PATH_INFO: `/v3/projects/905e399c9b6c4b61a626332780306517` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:55.096063 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed55ff9-0f63-47a4-be3b-de8144cb2393 admin admin] RBAC: Authorizing `identity:delete_project(project_id=905e399c9b6c4b61a626332780306517)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:55.098696 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed55ff9-0f63-47a4-be3b-de8144cb2393 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:55.212379 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-2ed55ff9-0f63-47a4-be3b-de8144cb2393 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '905e399c9b6c4b61a626332780306517', 'request_id': 'req-2ed55ff9-0f63-47a4-be3b-de8144cb2393'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:55.311426 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-2ed55ff9-0f63-47a4-be3b-de8144cb2393 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '905e399c9b6c4b61a626332780306517', 'request_id': 'req-2ed55ff9-0f63-47a4-be3b-de8144cb2393'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:55.319945 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-2ed55ff9-0f63-47a4-be3b-de8144cb2393 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '905e399c9b6c4b61a626332780306517', 'request_id': 'req-2ed55ff9-0f63-47a4-be3b-de8144cb2393'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:55.330574 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-2ed55ff9-0f63-47a4-be3b-de8144cb2393 admin admin] The token cache is being invalidate because project 905e399c9b6c4b61a626332780306517 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:37:55.330860 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-2ed55ff9-0f63-47a4-be3b-de8144cb2393 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:55.332060 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-2ed55ff9-0f63-47a4-be3b-de8144cb2393 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:55.333645 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 999/1990] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jul 15 19:37:55 2022] DELETE /identity/v3/projects/905e399c9b6c4b61a626332780306517 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:37:55.337886 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c9edda-8ac2-47c2-a0c0-6138973441d8 service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:55.349952 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c9edda-8ac2-47c2-a0c0-6138973441d8 service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:55.413025 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571674b8-f536-4826-bb18-f34b88840c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:55.414059 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571674b8-f536-4826-bb18-f34b88840c40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:55.414413 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571674b8-f536-4826-bb18-f34b88840c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:55.414788 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571674b8-f536-4826-bb18-f34b88840c40 admin admin] PATH_INFO: `/v3/users/7b849dfbe38c4825a1f12cca102cfde9` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:55.416093 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571674b8-f536-4826-bb18-f34b88840c40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b849dfbe38c4825a1f12cca102cfde9)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:55.418131 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571674b8-f536-4826-bb18-f34b88840c40 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:55.652854 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-571674b8-f536-4826-bb18-f34b88840c40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b849dfbe38c4825a1f12cca102cfde9'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:55.686999 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-571674b8-f536-4826-bb18-f34b88840c40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b849dfbe38c4825a1f12cca102cfde9'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:55.760578 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-571674b8-f536-4826-bb18-f34b88840c40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b849dfbe38c4825a1f12cca102cfde9'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:55.761870 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-571674b8-f536-4826-bb18-f34b88840c40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b849dfbe38c4825a1f12cca102cfde9'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:55.773050 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-571674b8-f536-4826-bb18-f34b88840c40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b849dfbe38c4825a1f12cca102cfde9'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:55.774791 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:37:55.783194 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 992/1991] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jul 15 19:37:55 2022] DELETE /identity/v3/users/7b849dfbe38c4825a1f12cca102cfde9 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:37:56.384158 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed55ff9-0f63-47a4-be3b-de8144cb2393 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:56.385697 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-2ed55ff9-0f63-47a4-be3b-de8144cb2393 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:37:56.401273 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed55ff9-0f63-47a4-be3b-de8144cb2393 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:56.490048 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b3fb07-59c4-4efc-9649-a51ccc11e1eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:56.491124 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b3fb07-59c4-4efc-9649-a51ccc11e1eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:56.491409 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b3fb07-59c4-4efc-9649-a51ccc11e1eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:56.491696 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b3fb07-59c4-4efc-9649-a51ccc11e1eb admin admin] PATH_INFO: `/v3/projects/ca6edeb3588a48a39e50017c90dfd4de` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:56.493243 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b3fb07-59c4-4efc-9649-a51ccc11e1eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca6edeb3588a48a39e50017c90dfd4de)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:56.495296 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b3fb07-59c4-4efc-9649-a51ccc11e1eb admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:56.705318 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:37:56.707182 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-42b3fb07-59c4-4efc-9649-a51ccc11e1eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca6edeb3588a48a39e50017c90dfd4de', 'request_id': 'req-42b3fb07-59c4-4efc-9649-a51ccc11e1eb'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:56.816766 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-42b3fb07-59c4-4efc-9649-a51ccc11e1eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca6edeb3588a48a39e50017c90dfd4de', 'request_id': 'req-42b3fb07-59c4-4efc-9649-a51ccc11e1eb'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:56.826980 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-42b3fb07-59c4-4efc-9649-a51ccc11e1eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca6edeb3588a48a39e50017c90dfd4de', 'request_id': 'req-42b3fb07-59c4-4efc-9649-a51ccc11e1eb'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:56.846296 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-42b3fb07-59c4-4efc-9649-a51ccc11e1eb admin admin] The token cache is being invalidate because project ca6edeb3588a48a39e50017c90dfd4de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:37:56.847669 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-42b3fb07-59c4-4efc-9649-a51ccc11e1eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:56.848423 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-42b3fb07-59c4-4efc-9649-a51ccc11e1eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:56.850337 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1000/1992] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jul 15 19:37:56 2022] DELETE /identity/v3/projects/ca6edeb3588a48a39e50017c90dfd4de => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:37:56.854622 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571674b8-f536-4826-bb18-f34b88840c40 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:56.856058 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-571674b8-f536-4826-bb18-f34b88840c40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:37:56.869171 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571674b8-f536-4826-bb18-f34b88840c40 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:56.928887 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7466e9ff-9bbb-4d8f-8efb-56f77920af4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:56.929788 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7466e9ff-9bbb-4d8f-8efb-56f77920af4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:56.930081 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7466e9ff-9bbb-4d8f-8efb-56f77920af4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:56.930351 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7466e9ff-9bbb-4d8f-8efb-56f77920af4c admin admin] PATH_INFO: `/v3/users/ed118dd44504404f880b91541e23ffac` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:56.931529 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7466e9ff-9bbb-4d8f-8efb-56f77920af4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed118dd44504404f880b91541e23ffac)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:56.934100 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7466e9ff-9bbb-4d8f-8efb-56f77920af4c admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:57.151067 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-7466e9ff-9bbb-4d8f-8efb-56f77920af4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed118dd44504404f880b91541e23ffac'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:57.171463 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-7466e9ff-9bbb-4d8f-8efb-56f77920af4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed118dd44504404f880b91541e23ffac'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:57.293935 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-7466e9ff-9bbb-4d8f-8efb-56f77920af4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed118dd44504404f880b91541e23ffac'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:57.294632 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-7466e9ff-9bbb-4d8f-8efb-56f77920af4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed118dd44504404f880b91541e23ffac'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:57.303735 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-7466e9ff-9bbb-4d8f-8efb-56f77920af4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed118dd44504404f880b91541e23ffac'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:57.309993 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 993/1993] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jul 15 19:37:56 2022] DELETE /identity/v3/users/ed118dd44504404f880b91541e23ffac => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:37:57.594142 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b3fb07-59c4-4efc-9649-a51ccc11e1eb admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:57.595623 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-42b3fb07-59c4-4efc-9649-a51ccc11e1eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:37:57.610636 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b3fb07-59c4-4efc-9649-a51ccc11e1eb admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:57.676822 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d343d8bf-2e93-4918-a6c5-c49e518b5f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:57.677709 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d343d8bf-2e93-4918-a6c5-c49e518b5f77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:57.678194 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d343d8bf-2e93-4918-a6c5-c49e518b5f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:57.678625 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d343d8bf-2e93-4918-a6c5-c49e518b5f77 admin admin] PATH_INFO: `/v3/users/e03ecc37b14d406987ca1ffc443a0874` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:57.682163 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d343d8bf-2e93-4918-a6c5-c49e518b5f77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e03ecc37b14d406987ca1ffc443a0874)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:57.684312 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d343d8bf-2e93-4918-a6c5-c49e518b5f77 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:57.790249 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7466e9ff-9bbb-4d8f-8efb-56f77920af4c admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:57.791991 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-7466e9ff-9bbb-4d8f-8efb-56f77920af4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:37:57.806991 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7466e9ff-9bbb-4d8f-8efb-56f77920af4c admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:57.811736 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b60c5c-dc85-4481-b10f-f943e6291e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:57.812619 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b60c5c-dc85-4481-b10f-f943e6291e2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:57.813467 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b60c5c-dc85-4481-b10f-f943e6291e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:57.813467 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b60c5c-dc85-4481-b10f-f943e6291e2e admin admin] PATH_INFO: `/v3/projects/deff5edeb3e148208182b715fd5015c6` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:57.814408 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b60c5c-dc85-4481-b10f-f943e6291e2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=deff5edeb3e148208182b715fd5015c6)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:57.816472 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b60c5c-dc85-4481-b10f-f943e6291e2e admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:57.873061 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-d343d8bf-2e93-4918-a6c5-c49e518b5f77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e03ecc37b14d406987ca1ffc443a0874'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:57.886848 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-d343d8bf-2e93-4918-a6c5-c49e518b5f77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e03ecc37b14d406987ca1ffc443a0874'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:57.965482 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-03b60c5c-dc85-4481-b10f-f943e6291e2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'deff5edeb3e148208182b715fd5015c6', 'request_id': 'req-03b60c5c-dc85-4481-b10f-f943e6291e2e'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:57.979070 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-d343d8bf-2e93-4918-a6c5-c49e518b5f77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e03ecc37b14d406987ca1ffc443a0874'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:57.980147 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-d343d8bf-2e93-4918-a6c5-c49e518b5f77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e03ecc37b14d406987ca1ffc443a0874'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:57.980755 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-d343d8bf-2e93-4918-a6c5-c49e518b5f77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e03ecc37b14d406987ca1ffc443a0874'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:57.993985 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1001/1994] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jul 15 19:37:57 2022] DELETE /identity/v3/users/e03ecc37b14d406987ca1ffc443a0874 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:37:57.998406 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa33ceec-cf0c-456e-9e97-91398a033d4d None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:57.998638 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa33ceec-cf0c-456e-9e97-91398a033d4d None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:57.998920 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa33ceec-cf0c-456e-9e97-91398a033d4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:58.017927 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-aa33ceec-cf0c-456e-9e97-91398a033d4d None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:37:58.019136 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-aa33ceec-cf0c-456e-9e97-91398a033d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:37:58.070152 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-03b60c5c-dc85-4481-b10f-f943e6291e2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'deff5edeb3e148208182b715fd5015c6', 'request_id': 'req-03b60c5c-dc85-4481-b10f-f943e6291e2e'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:58.076563 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-03b60c5c-dc85-4481-b10f-f943e6291e2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'deff5edeb3e148208182b715fd5015c6', 'request_id': 'req-03b60c5c-dc85-4481-b10f-f943e6291e2e'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:58.078320 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1002/1995] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:37:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:37:58.088391 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa33ceec-cf0c-456e-9e97-91398a033d4d None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:58.088579 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:37:58.089865 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-aa33ceec-cf0c-456e-9e97-91398a033d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:37:58.092909 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-03b60c5c-dc85-4481-b10f-f943e6291e2e admin admin] The token cache is being invalidate because project deff5edeb3e148208182b715fd5015c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:37:58.093244 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-03b60c5c-dc85-4481-b10f-f943e6291e2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:58.093938 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-03b60c5c-dc85-4481-b10f-f943e6291e2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:58.095615 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 994/1996] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jul 15 19:37:57 2022] DELETE /identity/v3/projects/deff5edeb3e148208182b715fd5015c6 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:37:58.103057 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d1314-45aa-4702-85e6-ebf1145e56ea None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:58.103348 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d1314-45aa-4702-85e6-ebf1145e56ea None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:58.103631 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d1314-45aa-4702-85e6-ebf1145e56ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:58.122651 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-413d1314-45aa-4702-85e6-ebf1145e56ea None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:37:58.123911 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-413d1314-45aa-4702-85e6-ebf1145e56ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:37:58.134663 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 995/1997] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:37:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:37:58.145222 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac3cde9-39b1-4687-8d8d-d44ef36cdc2d None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:58.146000 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac3cde9-39b1-4687-8d8d-d44ef36cdc2d None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:58.148293 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac3cde9-39b1-4687-8d8d-d44ef36cdc2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:58.153140 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa33ceec-cf0c-456e-9e97-91398a033d4d None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:58.165701 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ab9991-8f6f-4c53-82eb-becb40fafca1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:58.166409 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ab9991-8f6f-4c53-82eb-becb40fafca1 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:58.166884 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ab9991-8f6f-4c53-82eb-becb40fafca1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:58.167163 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ab9991-8f6f-4c53-82eb-becb40fafca1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:58.167292 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-fac3cde9-39b1-4687-8d8d-d44ef36cdc2d None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:37:58.168490 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-fac3cde9-39b1-4687-8d8d-d44ef36cdc2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:37:58.168933 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-f4ab9991-8f6f-4c53-82eb-becb40fafca1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:37:58.176564 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 996/1998] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:37:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:37:58.180927 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac3cde9-39b1-4687-8d8d-d44ef36cdc2d None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:58.181889 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ab9991-8f6f-4c53-82eb-becb40fafca1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:58.183340 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ab9991-8f6f-4c53-82eb-becb40fafca1 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:58.190444 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac3cde9-39b1-4687-8d8d-d44ef36cdc2d None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:58.194597 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1003/1999] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:37:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 19:37:58.195925 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d03adb-3a40-4c04-b800-6e2a9ed4cff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:58.196613 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d03adb-3a40-4c04-b800-6e2a9ed4cff8 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:58.196990 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d03adb-3a40-4c04-b800-6e2a9ed4cff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:58.197983 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d03adb-3a40-4c04-b800-6e2a9ed4cff8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:58.198780 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d03adb-3a40-4c04-b800-6e2a9ed4cff8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:37:58.199099 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d03adb-3a40-4c04-b800-6e2a9ed4cff8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:58.200779 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d03adb-3a40-4c04-b800-6e2a9ed4cff8 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:58.214377 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 997/2000] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jul 15 19:37:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:37:58.219061 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ab9991-8f6f-4c53-82eb-becb40fafca1 service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:58.230223 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ab9991-8f6f-4c53-82eb-becb40fafca1 service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:58.235542 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf30f5c9-d7fd-4f11-9127-b95bd69e7d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:58.236212 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf30f5c9-d7fd-4f11-9127-b95bd69e7d89 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:58.236499 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf30f5c9-d7fd-4f11-9127-b95bd69e7d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:58.236776 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf30f5c9-d7fd-4f11-9127-b95bd69e7d89 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:58.237493 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf30f5c9-d7fd-4f11-9127-b95bd69e7d89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:58.239592 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf30f5c9-d7fd-4f11-9127-b95bd69e7d89 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:58.296817 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:37:58.336686 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1004/2001] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jul 15 19:37:58 2022] POST /identity/v3/projects => generated 377 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:37:58.340856 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d03adb-3a40-4c04-b800-6e2a9ed4cff8 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:58.358115 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d03adb-3a40-4c04-b800-6e2a9ed4cff8 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:58.431304 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b528105-c121-42f2-a687-728f76c1864f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:58.432068 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b528105-c121-42f2-a687-728f76c1864f admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:58.432417 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b528105-c121-42f2-a687-728f76c1864f admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:58.432758 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b528105-c121-42f2-a687-728f76c1864f admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:58.433670 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b528105-c121-42f2-a687-728f76c1864f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:58.435811 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b528105-c121-42f2-a687-728f76c1864f admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:58.555617 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 998/2002] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jul 15 19:37:58 2022] POST /identity/v3/users => generated 289 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:37:58.561524 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf30f5c9-d7fd-4f11-9127-b95bd69e7d89 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:58.572908 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf30f5c9-d7fd-4f11-9127-b95bd69e7d89 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:58.581648 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677d63b5-f6ca-4566-a766-c51115a27f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:58.582386 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677d63b5-f6ca-4566-a766-c51115a27f18 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:58.582664 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677d63b5-f6ca-4566-a766-c51115a27f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:58.582942 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677d63b5-f6ca-4566-a766-c51115a27f18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:58.583681 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677d63b5-f6ca-4566-a766-c51115a27f18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:58.585600 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677d63b5-f6ca-4566-a766-c51115a27f18 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:58.596783 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1005/2003] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:37:58 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:37:58.601029 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b528105-c121-42f2-a687-728f76c1864f admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:58.611111 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b528105-c121-42f2-a687-728f76c1864f admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:58.615781 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0141878b-6748-42a8-a539-696c7e12cd35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:58.616440 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0141878b-6748-42a8-a539-696c7e12cd35 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:58.616721 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0141878b-6748-42a8-a539-696c7e12cd35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:58.616995 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0141878b-6748-42a8-a539-696c7e12cd35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:58.617841 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0141878b-6748-42a8-a539-696c7e12cd35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:58.619455 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0141878b-6748-42a8-a539-696c7e12cd35 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:58.630544 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 999/2004] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:37:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:37:58.635242 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677d63b5-f6ca-4566-a766-c51115a27f18 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:58.645848 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677d63b5-f6ca-4566-a766-c51115a27f18 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:58.650910 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb4c22e-2b23-4ee9-9225-fbd5d229e9b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:58.651800 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb4c22e-2b23-4ee9-9225-fbd5d229e9b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:58.656573 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb4c22e-2b23-4ee9-9225-fbd5d229e9b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:58.656573 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb4c22e-2b23-4ee9-9225-fbd5d229e9b6 admin admin] PATH_INFO: `/v3/projects/ef77f530825441aa8a7cd3113b39670e/users/79db73eeecb24588b34950fd0b4e828a/roles/28763c6e70094116b3a4b2895895fa58` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:58.660027 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0141878b-6748-42a8-a539-696c7e12cd35 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:58.662372 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0141878b-6748-42a8-a539-696c7e12cd35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:37:58.668277 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb4c22e-2b23-4ee9-9225-fbd5d229e9b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef77f530825441aa8a7cd3113b39670e, user_id=79db73eeecb24588b34950fd0b4e828a, role_id=28763c6e70094116b3a4b2895895fa58)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:58.671045 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb4c22e-2b23-4ee9-9225-fbd5d229e9b6 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:58.691340 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0141878b-6748-42a8-a539-696c7e12cd35 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:58.691340 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3e46c5-5c9a-4c97-b17e-3b00d1e1f2c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:58.692097 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3e46c5-5c9a-4c97-b17e-3b00d1e1f2c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:58.692389 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3e46c5-5c9a-4c97-b17e-3b00d1e1f2c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:58.692669 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3e46c5-5c9a-4c97-b17e-3b00d1e1f2c2 admin admin] PATH_INFO: `/v3/projects/670a6f26f92b477f98168f4340e0176b` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:58.694033 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3e46c5-5c9a-4c97-b17e-3b00d1e1f2c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=670a6f26f92b477f98168f4340e0176b)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:58.696024 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3e46c5-5c9a-4c97-b17e-3b00d1e1f2c2 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:58.785149 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1006/2005] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:37:58 2022] PUT /identity/v3/projects/ef77f530825441aa8a7cd3113b39670e/users/79db73eeecb24588b34950fd0b4e828a/roles/28763c6e70094116b3a4b2895895fa58 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:37:58.796434 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e672b5-afd7-45ae-8c48-bdf94a38c2f0 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:58.806596 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e672b5-afd7-45ae-8c48-bdf94a38c2f0 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:58.806596 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e672b5-afd7-45ae-8c48-bdf94a38c2f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:58.842739 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-b3e672b5-afd7-45ae-8c48-bdf94a38c2f0 None None] MFA Rules not processed for user `79db73eeecb24588b34950fd0b4e828a`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:37:58.843546 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-b3e672b5-afd7-45ae-8c48-bdf94a38c2f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:37:58.863650 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9d3e46c5-5c9a-4c97-b17e-3b00d1e1f2c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '670a6f26f92b477f98168f4340e0176b', 'request_id': 'req-9d3e46c5-5c9a-4c97-b17e-3b00d1e1f2c2'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:58.939802 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1007/2006] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:37:58 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:37:58.952587 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e672b5-afd7-45ae-8c48-bdf94a38c2f0 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:58.954630 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-b3e672b5-afd7-45ae-8c48-bdf94a38c2f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:37:58.994534 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9d3e46c5-5c9a-4c97-b17e-3b00d1e1f2c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '670a6f26f92b477f98168f4340e0176b', 'request_id': 'req-9d3e46c5-5c9a-4c97-b17e-3b00d1e1f2c2'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:59.002682 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9d3e46c5-5c9a-4c97-b17e-3b00d1e1f2c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '670a6f26f92b477f98168f4340e0176b', 'request_id': 'req-9d3e46c5-5c9a-4c97-b17e-3b00d1e1f2c2'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:59.018942 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9d3e46c5-5c9a-4c97-b17e-3b00d1e1f2c2 admin admin] The token cache is being invalidate because project 670a6f26f92b477f98168f4340e0176b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:37:59.019579 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9d3e46c5-5c9a-4c97-b17e-3b00d1e1f2c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:59.021253 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9d3e46c5-5c9a-4c97-b17e-3b00d1e1f2c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:59.024569 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1000/2007] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jul 15 19:37:58 2022] DELETE /identity/v3/projects/670a6f26f92b477f98168f4340e0176b => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:37:59.028533 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3e46c5-5c9a-4c97-b17e-3b00d1e1f2c2 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:37:59.030191 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9d3e46c5-5c9a-4c97-b17e-3b00d1e1f2c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:37:59.049257 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e672b5-afd7-45ae-8c48-bdf94a38c2f0 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:59.054131 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3439ca6-0dbd-4083-91bb-e6603cdad395 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04b5b06b0015450a95a28bbcc757cc8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:59.054818 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3439ca6-0dbd-4083-91bb-e6603cdad395 service glance] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:59.055081 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3439ca6-0dbd-4083-91bb-e6603cdad395 service glance] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:59.055361 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3439ca6-0dbd-4083-91bb-e6603cdad395 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:59.057281 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-f3439ca6-0dbd-4083-91bb-e6603cdad395 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:37:59.070902 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3439ca6-0dbd-4083-91bb-e6603cdad395 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:59.072330 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3439ca6-0dbd-4083-91bb-e6603cdad395 service glance] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:59.082122 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1008/2008] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:37:58 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 19:37:59.104359 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3e46c5-5c9a-4c97-b17e-3b00d1e1f2c2 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:37:59.109947 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e14b91-b1fc-4589-8c40-6dbc175ebeb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:37:59.116983 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e14b91-b1fc-4589-8c40-6dbc175ebeb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:37:59.116983 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e14b91-b1fc-4589-8c40-6dbc175ebeb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:37:59.116983 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e14b91-b1fc-4589-8c40-6dbc175ebeb1 admin admin] PATH_INFO: `/v3/users/0fb82d64c34a46ca8a469777735e2aad` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:37:59.116983 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e14b91-b1fc-4589-8c40-6dbc175ebeb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fb82d64c34a46ca8a469777735e2aad)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:37:59.116983 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e14b91-b1fc-4589-8c40-6dbc175ebeb1 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:37:59.411660 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-f4e14b91-b1fc-4589-8c40-6dbc175ebeb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fb82d64c34a46ca8a469777735e2aad'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:59.431937 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-f4e14b91-b1fc-4589-8c40-6dbc175ebeb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fb82d64c34a46ca8a469777735e2aad'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:59.526171 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:37:59.557507 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-f4e14b91-b1fc-4589-8c40-6dbc175ebeb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fb82d64c34a46ca8a469777735e2aad'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:59.570756 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-f4e14b91-b1fc-4589-8c40-6dbc175ebeb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fb82d64c34a46ca8a469777735e2aad'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:59.582296 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-f4e14b91-b1fc-4589-8c40-6dbc175ebeb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fb82d64c34a46ca8a469777735e2aad'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:37:59.588762 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1001/2009] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jul 15 19:37:59 2022] DELETE /identity/v3/users/0fb82d64c34a46ca8a469777735e2aad => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:37:59.663347 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:38:00.534776 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3439ca6-0dbd-4083-91bb-e6603cdad395 service glance] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:00.547640 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3439ca6-0dbd-4083-91bb-e6603cdad395 service glance] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:00.629624 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac912859-f814-4dea-b2d1-9f012c8dfcf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:00.630555 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac912859-f814-4dea-b2d1-9f012c8dfcf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:00.630841 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac912859-f814-4dea-b2d1-9f012c8dfcf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:00.631115 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac912859-f814-4dea-b2d1-9f012c8dfcf4 admin admin] PATH_INFO: `/v3/projects/5837828c54804b74b96dfbfe8acb7207` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:00.632830 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac912859-f814-4dea-b2d1-9f012c8dfcf4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5837828c54804b74b96dfbfe8acb7207)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:00.635447 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac912859-f814-4dea-b2d1-9f012c8dfcf4 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:00.769179 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-ac912859-f814-4dea-b2d1-9f012c8dfcf4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5837828c54804b74b96dfbfe8acb7207', 'request_id': 'req-ac912859-f814-4dea-b2d1-9f012c8dfcf4'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:00.865222 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-ac912859-f814-4dea-b2d1-9f012c8dfcf4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5837828c54804b74b96dfbfe8acb7207', 'request_id': 'req-ac912859-f814-4dea-b2d1-9f012c8dfcf4'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:00.872393 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-ac912859-f814-4dea-b2d1-9f012c8dfcf4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5837828c54804b74b96dfbfe8acb7207', 'request_id': 'req-ac912859-f814-4dea-b2d1-9f012c8dfcf4'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:00.885480 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-ac912859-f814-4dea-b2d1-9f012c8dfcf4 admin admin] The token cache is being invalidate because project 5837828c54804b74b96dfbfe8acb7207 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:38:00.885696 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-ac912859-f814-4dea-b2d1-9f012c8dfcf4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:00.886440 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-ac912859-f814-4dea-b2d1-9f012c8dfcf4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:00.888061 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1009/2010] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jul 15 19:38:00 2022] DELETE /identity/v3/projects/5837828c54804b74b96dfbfe8acb7207 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:38:00.893003 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fd060b-e038-47a8-bd0f-63075dde7963 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:00.893504 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fd060b-e038-47a8-bd0f-63075dde7963 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:00.894164 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fd060b-e038-47a8-bd0f-63075dde7963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:00.914961 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-29fd060b-e038-47a8-bd0f-63075dde7963 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:38:00.916472 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-29fd060b-e038-47a8-bd0f-63075dde7963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:00.993949 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1002/2011] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:38:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:38:00.995885 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9824d2a2-963a-4907-8e48-a772019d9ad4 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:00.996177 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9824d2a2-963a-4907-8e48-a772019d9ad4 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:00.996581 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9824d2a2-963a-4907-8e48-a772019d9ad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:01.026141 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-9824d2a2-963a-4907-8e48-a772019d9ad4 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:38:01.028108 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-9824d2a2-963a-4907-8e48-a772019d9ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:01.040572 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1010/2012] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:38:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:38:01.044684 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fd060b-e038-47a8-bd0f-63075dde7963 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:01.062510 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fd060b-e038-47a8-bd0f-63075dde7963 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:01.069943 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab52f7bb-55bf-4677-b8c4-2031a3ece364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:01.070715 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab52f7bb-55bf-4677-b8c4-2031a3ece364 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:01.071023 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab52f7bb-55bf-4677-b8c4-2031a3ece364 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:01.071333 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab52f7bb-55bf-4677-b8c4-2031a3ece364 admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:01.072500 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab52f7bb-55bf-4677-b8c4-2031a3ece364 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:38:01.072997 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab52f7bb-55bf-4677-b8c4-2031a3ece364 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:01.074792 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab52f7bb-55bf-4677-b8c4-2031a3ece364 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:01.088815 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1003/2013] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jul 15 19:38:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:38:01.093177 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9824d2a2-963a-4907-8e48-a772019d9ad4 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:01.105535 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9824d2a2-963a-4907-8e48-a772019d9ad4 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:01.110445 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd97546-9586-410e-b65d-dec38152e2ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:01.111358 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd97546-9586-410e-b65d-dec38152e2ef admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:01.111489 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd97546-9586-410e-b65d-dec38152e2ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:01.112592 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd97546-9586-410e-b65d-dec38152e2ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:01.112922 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd97546-9586-410e-b65d-dec38152e2ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:01.115054 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd97546-9586-410e-b65d-dec38152e2ef admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:01.243043 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1011/2014] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jul 15 19:38:01 2022] POST /identity/v3/projects => generated 367 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:38:01.244710 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab52f7bb-55bf-4677-b8c4-2031a3ece364 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:01.255526 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab52f7bb-55bf-4677-b8c4-2031a3ece364 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:01.314641 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a325ada9-20a7-41f5-a73c-9cce86a8e7ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:01.315496 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a325ada9-20a7-41f5-a73c-9cce86a8e7ab admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:01.316002 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a325ada9-20a7-41f5-a73c-9cce86a8e7ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:01.316380 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a325ada9-20a7-41f5-a73c-9cce86a8e7ab admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:01.317104 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a325ada9-20a7-41f5-a73c-9cce86a8e7ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:01.319379 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a325ada9-20a7-41f5-a73c-9cce86a8e7ab admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:01.323628 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:38:01.450220 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1004/2015] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jul 15 19:38:01 2022] POST /identity/v3/users => generated 284 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:38:01.454904 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd97546-9586-410e-b65d-dec38152e2ef admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:01.466192 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd97546-9586-410e-b65d-dec38152e2ef admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:01.470890 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1409ca9f-c0e7-4f4d-a8ba-af56c3d2eeb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:01.471585 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1409ca9f-c0e7-4f4d-a8ba-af56c3d2eeb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:01.471867 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1409ca9f-c0e7-4f4d-a8ba-af56c3d2eeb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:01.472141 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1409ca9f-c0e7-4f4d-a8ba-af56c3d2eeb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:01.473037 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1409ca9f-c0e7-4f4d-a8ba-af56c3d2eeb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:01.474667 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1409ca9f-c0e7-4f4d-a8ba-af56c3d2eeb7 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:01.486169 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1012/2016] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:38:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:38:01.491989 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a325ada9-20a7-41f5-a73c-9cce86a8e7ab admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:01.503615 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a325ada9-20a7-41f5-a73c-9cce86a8e7ab admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:01.508829 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40139519-4b56-4ea4-a582-0fb03d563329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:01.509499 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40139519-4b56-4ea4-a582-0fb03d563329 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:01.509837 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40139519-4b56-4ea4-a582-0fb03d563329 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:01.510120 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40139519-4b56-4ea4-a582-0fb03d563329 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:01.510855 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40139519-4b56-4ea4-a582-0fb03d563329 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:01.512485 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40139519-4b56-4ea4-a582-0fb03d563329 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:01.523482 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1005/2017] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:38:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 19:38:01.527727 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1409ca9f-c0e7-4f4d-a8ba-af56c3d2eeb7 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:01.534438 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:38:01.542556 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1409ca9f-c0e7-4f4d-a8ba-af56c3d2eeb7 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:01.554171 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dfc2d3-40fa-4fa6-acce-deb96810c15a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:01.555214 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dfc2d3-40fa-4fa6-acce-deb96810c15a admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:01.555501 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dfc2d3-40fa-4fa6-acce-deb96810c15a admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:01.555779 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dfc2d3-40fa-4fa6-acce-deb96810c15a admin admin] PATH_INFO: `/v3/projects/df23fee98cce4b43b1dad88e25185677/users/8b665a04e41b493daccf5e84f0802542/roles/28763c6e70094116b3a4b2895895fa58` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:01.573301 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dfc2d3-40fa-4fa6-acce-deb96810c15a admin admin] RBAC: Authorizing `identity:create_grant(project_id=df23fee98cce4b43b1dad88e25185677, user_id=8b665a04e41b493daccf5e84f0802542, role_id=28763c6e70094116b3a4b2895895fa58)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:01.576006 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dfc2d3-40fa-4fa6-acce-deb96810c15a admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:01.688125 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1013/2018] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:38:01 2022] PUT /identity/v3/projects/df23fee98cce4b43b1dad88e25185677/users/8b665a04e41b493daccf5e84f0802542/roles/28763c6e70094116b3a4b2895895fa58 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:38:01.700621 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6f2c0e-1629-477d-aefe-f568828795b3 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:01.700904 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6f2c0e-1629-477d-aefe-f568828795b3 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:01.701093 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6f2c0e-1629-477d-aefe-f568828795b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:01.729198 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-db6f2c0e-1629-477d-aefe-f568828795b3 None None] MFA Rules not processed for user `8b665a04e41b493daccf5e84f0802542`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:38:01.730501 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-db6f2c0e-1629-477d-aefe-f568828795b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:01.801750 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1006/2019] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:38:01 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:38:01.803066 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dfc2d3-40fa-4fa6-acce-deb96810c15a admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:01.867002 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dfc2d3-40fa-4fa6-acce-deb96810c15a admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:01.871732 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9567163-8fed-4743-8682-924a3dd97bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:01.872507 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9567163-8fed-4743-8682-924a3dd97bf4 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:01.872827 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9567163-8fed-4743-8682-924a3dd97bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:01.873105 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9567163-8fed-4743-8682-924a3dd97bf4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:01.873825 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9567163-8fed-4743-8682-924a3dd97bf4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:01.876272 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9567163-8fed-4743-8682-924a3dd97bf4 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:01.967859 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1014/2020] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jul 15 19:38:01 2022] POST /identity/v3/projects => generated 369 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:38:01.971773 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6f2c0e-1629-477d-aefe-f568828795b3 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:01.983183 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6f2c0e-1629-477d-aefe-f568828795b3 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:02.040823 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aad611-352e-4d5a-93b3-5608cb87d1a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:02.041506 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aad611-352e-4d5a-93b3-5608cb87d1a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:02.041886 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aad611-352e-4d5a-93b3-5608cb87d1a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:02.042143 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aad611-352e-4d5a-93b3-5608cb87d1a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:02.042850 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8aad611-352e-4d5a-93b3-5608cb87d1a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:02.044781 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8aad611-352e-4d5a-93b3-5608cb87d1a0 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:02.158346 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1007/2021] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jul 15 19:38:01 2022] POST /identity/v3/users => generated 291 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:38:02.162400 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9567163-8fed-4743-8682-924a3dd97bf4 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:02.176415 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9567163-8fed-4743-8682-924a3dd97bf4 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:02.182077 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fd7431-a33a-4742-b3fa-1d038bfa5f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:02.182939 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fd7431-a33a-4742-b3fa-1d038bfa5f73 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:02.183293 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fd7431-a33a-4742-b3fa-1d038bfa5f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:02.183639 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fd7431-a33a-4742-b3fa-1d038bfa5f73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:02.184477 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fd7431-a33a-4742-b3fa-1d038bfa5f73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:02.186813 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fd7431-a33a-4742-b3fa-1d038bfa5f73 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:02.198668 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1015/2022] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:38:02 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:38:02.203241 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aad611-352e-4d5a-93b3-5608cb87d1a0 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:02.214898 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aad611-352e-4d5a-93b3-5608cb87d1a0 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:02.220724 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4c24e0-61b0-42d9-aaad-daab2ff9d5a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:02.222124 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4c24e0-61b0-42d9-aaad-daab2ff9d5a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:02.225161 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4c24e0-61b0-42d9-aaad-daab2ff9d5a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:02.225473 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4c24e0-61b0-42d9-aaad-daab2ff9d5a6 admin admin] PATH_INFO: `/v3/projects/1b858f31a90144d48ead15202cd82968/users/7dcdc8b2520045128c39f4074c461581/roles/4318f86ba9f74cd4a0bef1c311276657` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:02.245747 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4c24e0-61b0-42d9-aaad-daab2ff9d5a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b858f31a90144d48ead15202cd82968, user_id=7dcdc8b2520045128c39f4074c461581, role_id=4318f86ba9f74cd4a0bef1c311276657)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:02.250549 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4c24e0-61b0-42d9-aaad-daab2ff9d5a6 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:02.396661 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1008/2023] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:38:02 2022] PUT /identity/v3/projects/1b858f31a90144d48ead15202cd82968/users/7dcdc8b2520045128c39f4074c461581/roles/4318f86ba9f74cd4a0bef1c311276657 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:38:02.400793 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fd7431-a33a-4742-b3fa-1d038bfa5f73 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:02.412218 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fd7431-a33a-4742-b3fa-1d038bfa5f73 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:02.470567 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8024ba8a-3182-4c1a-9eb5-e0719cb9321b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:02.471308 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8024ba8a-3182-4c1a-9eb5-e0719cb9321b admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:02.472053 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8024ba8a-3182-4c1a-9eb5-e0719cb9321b admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:02.472359 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8024ba8a-3182-4c1a-9eb5-e0719cb9321b admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:02.473081 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8024ba8a-3182-4c1a-9eb5-e0719cb9321b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:02.474844 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8024ba8a-3182-4c1a-9eb5-e0719cb9321b admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:02.485373 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1016/2024] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:38:02 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:38:02.490314 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4c24e0-61b0-42d9-aaad-daab2ff9d5a6 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:02.500569 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4c24e0-61b0-42d9-aaad-daab2ff9d5a6 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:02.507500 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ec4985-a1df-410a-8299-17a1f5e7773d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:02.508703 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec4985-a1df-410a-8299-17a1f5e7773d admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:02.508987 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec4985-a1df-410a-8299-17a1f5e7773d admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:02.510591 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec4985-a1df-410a-8299-17a1f5e7773d admin admin] PATH_INFO: `/v3/projects/1b858f31a90144d48ead15202cd82968/users/7dcdc8b2520045128c39f4074c461581/roles/4318f86ba9f74cd4a0bef1c311276657` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:02.511478 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ec4985-a1df-410a-8299-17a1f5e7773d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b858f31a90144d48ead15202cd82968, user_id=7dcdc8b2520045128c39f4074c461581, role_id=4318f86ba9f74cd4a0bef1c311276657)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:02.515451 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ec4985-a1df-410a-8299-17a1f5e7773d admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:02.527646 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1009/2025] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:38:02 2022] PUT /identity/v3/projects/1b858f31a90144d48ead15202cd82968/users/7dcdc8b2520045128c39f4074c461581/roles/4318f86ba9f74cd4a0bef1c311276657 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:38:02.539243 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b93935-b848-497c-9359-2db1ca872a01 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:02.539620 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b93935-b848-497c-9359-2db1ca872a01 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:02.539972 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b93935-b848-497c-9359-2db1ca872a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:02.567030 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-62b93935-b848-497c-9359-2db1ca872a01 None None] MFA Rules not processed for user `7dcdc8b2520045128c39f4074c461581`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:38:02.570401 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-62b93935-b848-497c-9359-2db1ca872a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:02.609981 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:38:02.637860 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1017/2026] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:38:02 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:38:02.650353 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ec4985-a1df-410a-8299-17a1f5e7773d admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:02.652133 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-89ec4985-a1df-410a-8299-17a1f5e7773d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:02.657484 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:38:02.713826 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ec4985-a1df-410a-8299-17a1f5e7773d admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:02.717674 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d04840-31ed-4a80-b9b7-e09fe2b810b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86c2df3d378490ca7ea497a73c19d40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:02.718358 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d04840-31ed-4a80-b9b7-e09fe2b810b0 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:02.718639 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d04840-31ed-4a80-b9b7-e09fe2b810b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:02.718912 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d04840-31ed-4a80-b9b7-e09fe2b810b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:02.731908 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d04840-31ed-4a80-b9b7-e09fe2b810b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:02.733405 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d04840-31ed-4a80-b9b7-e09fe2b810b0 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:02.743555 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1010/2027] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:38:02 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:38:02.879676 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b93935-b848-497c-9359-2db1ca872a01 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:02.881155 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-62b93935-b848-497c-9359-2db1ca872a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:02.894809 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b93935-b848-497c-9359-2db1ca872a01 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:02.898456 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee129ef3-bc6b-43bc-a64d-06578366a986 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86c2df3d378490ca7ea497a73c19d40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:02.899132 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee129ef3-bc6b-43bc-a64d-06578366a986 service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:02.899416 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee129ef3-bc6b-43bc-a64d-06578366a986 service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:02.899705 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee129ef3-bc6b-43bc-a64d-06578366a986 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:02.958160 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee129ef3-bc6b-43bc-a64d-06578366a986 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:02.959603 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee129ef3-bc6b-43bc-a64d-06578366a986 service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:02.968888 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1018/2028] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:38:02 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:38:08.591806 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d04840-31ed-4a80-b9b7-e09fe2b810b0 service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:08.658023 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d04840-31ed-4a80-b9b7-e09fe2b810b0 service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:08.662696 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f44184-1bf0-44d8-a584-34913d86992a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:08.663577 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f44184-1bf0-44d8-a584-34913d86992a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:08.663858 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f44184-1bf0-44d8-a584-34913d86992a admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:08.664138 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f44184-1bf0-44d8-a584-34913d86992a admin admin] PATH_INFO: `/v3/users/8b665a04e41b493daccf5e84f0802542` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:08.665478 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f44184-1bf0-44d8-a584-34913d86992a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b665a04e41b493daccf5e84f0802542)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:08.667436 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f44184-1bf0-44d8-a584-34913d86992a admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:08.856484 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b8f44184-1bf0-44d8-a584-34913d86992a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b665a04e41b493daccf5e84f0802542'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:08.869127 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b8f44184-1bf0-44d8-a584-34913d86992a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b665a04e41b493daccf5e84f0802542'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:08.973923 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b8f44184-1bf0-44d8-a584-34913d86992a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b665a04e41b493daccf5e84f0802542'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:08.974540 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b8f44184-1bf0-44d8-a584-34913d86992a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b665a04e41b493daccf5e84f0802542'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:08.982285 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b8f44184-1bf0-44d8-a584-34913d86992a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b665a04e41b493daccf5e84f0802542'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:08.989603 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1011/2029] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jul 15 19:38:08 2022] DELETE /identity/v3/users/8b665a04e41b493daccf5e84f0802542 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:38:08.994517 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818b6ddc-869c-4483-a90b-b7437005d4fe None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:08.994977 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818b6ddc-869c-4483-a90b-b7437005d4fe None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:08.995133 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818b6ddc-869c-4483-a90b-b7437005d4fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:09.012913 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-818b6ddc-869c-4483-a90b-b7437005d4fe None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:38:09.014166 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-818b6ddc-869c-4483-a90b-b7437005d4fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:09.089395 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1019/2030] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:38:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:38:09.100326 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f44184-1bf0-44d8-a584-34913d86992a admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:09.101734 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b8f44184-1bf0-44d8-a584-34913d86992a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:09.139116 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:38:09.173296 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f44184-1bf0-44d8-a584-34913d86992a admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:09.179119 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a685472-6396-4c7e-95c0-95e1b21b90de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:09.179363 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a685472-6396-4c7e-95c0-95e1b21b90de service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:09.179794 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a685472-6396-4c7e-95c0-95e1b21b90de service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:09.180078 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a685472-6396-4c7e-95c0-95e1b21b90de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:09.190355 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a685472-6396-4c7e-95c0-95e1b21b90de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:09.192748 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a685472-6396-4c7e-95c0-95e1b21b90de service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:09.206183 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1012/2031] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:38:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:38:09.718774 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818b6ddc-869c-4483-a90b-b7437005d4fe None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:09.721569 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-818b6ddc-869c-4483-a90b-b7437005d4fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:09.771505 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818b6ddc-869c-4483-a90b-b7437005d4fe None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:09.790970 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238f3035-b898-4046-afe1-95b7b4100786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:09.793410 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238f3035-b898-4046-afe1-95b7b4100786 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:09.793792 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238f3035-b898-4046-afe1-95b7b4100786 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:09.794157 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238f3035-b898-4046-afe1-95b7b4100786 admin admin] PATH_INFO: `/v3/projects/df23fee98cce4b43b1dad88e25185677` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:09.795527 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238f3035-b898-4046-afe1-95b7b4100786 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df23fee98cce4b43b1dad88e25185677)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:09.798919 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238f3035-b898-4046-afe1-95b7b4100786 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:09.924432 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-238f3035-b898-4046-afe1-95b7b4100786 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df23fee98cce4b43b1dad88e25185677', 'request_id': 'req-238f3035-b898-4046-afe1-95b7b4100786'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:09.999637 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-238f3035-b898-4046-afe1-95b7b4100786 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df23fee98cce4b43b1dad88e25185677', 'request_id': 'req-238f3035-b898-4046-afe1-95b7b4100786'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:10.009234 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-238f3035-b898-4046-afe1-95b7b4100786 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df23fee98cce4b43b1dad88e25185677', 'request_id': 'req-238f3035-b898-4046-afe1-95b7b4100786'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:10.017397 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:38:10.022574 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-238f3035-b898-4046-afe1-95b7b4100786 admin admin] The token cache is being invalidate because project df23fee98cce4b43b1dad88e25185677 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:38:10.022930 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-238f3035-b898-4046-afe1-95b7b4100786 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:10.023746 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-238f3035-b898-4046-afe1-95b7b4100786 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:10.025570 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1020/2032] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jul 15 19:38:09 2022] DELETE /identity/v3/projects/df23fee98cce4b43b1dad88e25185677 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:38:10.029560 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a685472-6396-4c7e-95c0-95e1b21b90de service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:10.040499 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a685472-6396-4c7e-95c0-95e1b21b90de service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:10.102573 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e11a48a-d878-4290-93de-69bc7392fd0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:10.103486 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e11a48a-d878-4290-93de-69bc7392fd0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:10.103852 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e11a48a-d878-4290-93de-69bc7392fd0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:10.104310 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e11a48a-d878-4290-93de-69bc7392fd0e admin admin] PATH_INFO: `/v3/users/7dcdc8b2520045128c39f4074c461581` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:10.105603 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e11a48a-d878-4290-93de-69bc7392fd0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7dcdc8b2520045128c39f4074c461581)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:10.107849 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e11a48a-d878-4290-93de-69bc7392fd0e admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:10.418962 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4e11a48a-d878-4290-93de-69bc7392fd0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7dcdc8b2520045128c39f4074c461581'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:10.433565 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4e11a48a-d878-4290-93de-69bc7392fd0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7dcdc8b2520045128c39f4074c461581'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:10.537134 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4e11a48a-d878-4290-93de-69bc7392fd0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7dcdc8b2520045128c39f4074c461581'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:10.538006 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4e11a48a-d878-4290-93de-69bc7392fd0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7dcdc8b2520045128c39f4074c461581'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:10.558693 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4e11a48a-d878-4290-93de-69bc7392fd0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7dcdc8b2520045128c39f4074c461581'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:10.574535 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1013/2033] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jul 15 19:38:10 2022] DELETE /identity/v3/users/7dcdc8b2520045128c39f4074c461581 => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:38:11.569899 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238f3035-b898-4046-afe1-95b7b4100786 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:11.572911 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-238f3035-b898-4046-afe1-95b7b4100786 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:11.629977 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238f3035-b898-4046-afe1-95b7b4100786 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:11.731236 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373c686e-459c-40bd-86ff-c036210cf706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:11.732117 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c686e-459c-40bd-86ff-c036210cf706 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:11.732644 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c686e-459c-40bd-86ff-c036210cf706 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:11.732998 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c686e-459c-40bd-86ff-c036210cf706 admin admin] PATH_INFO: `/v3/projects/1b858f31a90144d48ead15202cd82968` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:11.734859 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373c686e-459c-40bd-86ff-c036210cf706 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b858f31a90144d48ead15202cd82968)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:11.738891 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373c686e-459c-40bd-86ff-c036210cf706 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:11.878996 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-373c686e-459c-40bd-86ff-c036210cf706 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b858f31a90144d48ead15202cd82968', 'request_id': 'req-373c686e-459c-40bd-86ff-c036210cf706'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:11.993392 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-373c686e-459c-40bd-86ff-c036210cf706 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b858f31a90144d48ead15202cd82968', 'request_id': 'req-373c686e-459c-40bd-86ff-c036210cf706'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:12.008826 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-373c686e-459c-40bd-86ff-c036210cf706 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b858f31a90144d48ead15202cd82968', 'request_id': 'req-373c686e-459c-40bd-86ff-c036210cf706'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:12.013455 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-373c686e-459c-40bd-86ff-c036210cf706 admin admin] The token cache is being invalidate because project 1b858f31a90144d48ead15202cd82968 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:38:12.013745 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-373c686e-459c-40bd-86ff-c036210cf706 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:12.014442 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-373c686e-459c-40bd-86ff-c036210cf706 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:12.016222 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1021/2034] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jul 15 19:38:11 2022] DELETE /identity/v3/projects/1b858f31a90144d48ead15202cd82968 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:38:12.021588 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d056b3-40fd-46c4-9d25-e3aa0bd9118e None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:12.022147 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d056b3-40fd-46c4-9d25-e3aa0bd9118e None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:12.022241 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d056b3-40fd-46c4-9d25-e3aa0bd9118e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:12.044495 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-99d056b3-40fd-46c4-9d25-e3aa0bd9118e None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:38:12.046385 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-99d056b3-40fd-46c4-9d25-e3aa0bd9118e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:12.159266 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:38:12.180117 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1014/2035] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:38:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:38:12.188012 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebc5d19-4e70-41f0-a9f0-b0b4544a5cdf None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:12.188295 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebc5d19-4e70-41f0-a9f0-b0b4544a5cdf None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:12.188578 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebc5d19-4e70-41f0-a9f0-b0b4544a5cdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:12.208894 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-7ebc5d19-4e70-41f0-a9f0-b0b4544a5cdf None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:38:12.210165 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-7ebc5d19-4e70-41f0-a9f0-b0b4544a5cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:12.218434 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1022/2036] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:38:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:38:12.227465 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d056b3-40fd-46c4-9d25-e3aa0bd9118e None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:12.237237 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d056b3-40fd-46c4-9d25-e3aa0bd9118e None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:12.241967 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a347eab0-56b9-442c-82cb-385495e91342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:12.242701 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a347eab0-56b9-442c-82cb-385495e91342 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:12.243326 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a347eab0-56b9-442c-82cb-385495e91342 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:12.243608 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a347eab0-56b9-442c-82cb-385495e91342 admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:12.244366 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a347eab0-56b9-442c-82cb-385495e91342 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:38:12.244749 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a347eab0-56b9-442c-82cb-385495e91342 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:12.246519 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a347eab0-56b9-442c-82cb-385495e91342 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:12.259084 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1015/2037] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jul 15 19:38:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:38:12.271332 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebc5d19-4e70-41f0-a9f0-b0b4544a5cdf None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:12.280667 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebc5d19-4e70-41f0-a9f0-b0b4544a5cdf None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:12.285375 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54abbdc8-62ea-4b3d-8baa-cfa0b9da0c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:12.286107 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54abbdc8-62ea-4b3d-8baa-cfa0b9da0c22 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:12.286393 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54abbdc8-62ea-4b3d-8baa-cfa0b9da0c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:12.286756 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54abbdc8-62ea-4b3d-8baa-cfa0b9da0c22 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:12.287456 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54abbdc8-62ea-4b3d-8baa-cfa0b9da0c22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:12.289386 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54abbdc8-62ea-4b3d-8baa-cfa0b9da0c22 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:12.397835 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1023/2038] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jul 15 19:38:12 2022] POST /identity/v3/projects => generated 365 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:38:12.402112 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a347eab0-56b9-442c-82cb-385495e91342 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:12.412907 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a347eab0-56b9-442c-82cb-385495e91342 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:12.474614 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37188875-02ec-419f-991b-a98d656829d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:12.475391 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37188875-02ec-419f-991b-a98d656829d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:12.475744 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37188875-02ec-419f-991b-a98d656829d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:12.476092 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37188875-02ec-419f-991b-a98d656829d3 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:12.476868 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37188875-02ec-419f-991b-a98d656829d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:12.478918 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37188875-02ec-419f-991b-a98d656829d3 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:12.498382 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdeb5b9-5f87-4564-b7d8-374a4a31c639 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:12.498650 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdeb5b9-5f87-4564-b7d8-374a4a31c639 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:12.499055 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdeb5b9-5f87-4564-b7d8-374a4a31c639 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:12.500511 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1024/2039] 10.222.0.33 () {54 vars in 921 bytes} [Fri Jul 15 19:38:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:38:12.542167 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1611f95-3d19-41b9-93dc-c433732ad5b8 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:12.542682 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1611f95-3d19-41b9-93dc-c433732ad5b8 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:12.542968 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1611f95-3d19-41b9-93dc-c433732ad5b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:12.566576 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-f1611f95-3d19-41b9-93dc-c433732ad5b8 None None] MFA Rules not processed for user `52a0fce231ba4785a685ac9898d07503`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:38:12.568100 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-f1611f95-3d19-41b9-93dc-c433732ad5b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:12.619849 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1016/2040] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jul 15 19:38:12 2022] POST /identity/v3/users => generated 283 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:38:12.620085 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:38:12.621696 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37188875-02ec-419f-991b-a98d656829d3 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:12.631857 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1025/2041] 10.222.0.33 () {58 vars in 1018 bytes} [Fri Jul 15 19:38:12 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:38:12.632218 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37188875-02ec-419f-991b-a98d656829d3 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:12.637267 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8168986-21e1-49ce-b26b-007b6e739009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:12.638108 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8168986-21e1-49ce-b26b-007b6e739009 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:12.638225 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8168986-21e1-49ce-b26b-007b6e739009 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:12.638509 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8168986-21e1-49ce-b26b-007b6e739009 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:12.639227 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8168986-21e1-49ce-b26b-007b6e739009 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:12.640832 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8168986-21e1-49ce-b26b-007b6e739009 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:12.643351 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1611f95-3d19-41b9-93dc-c433732ad5b8 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:12.644991 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-f1611f95-3d19-41b9-93dc-c433732ad5b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:12.653570 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1017/2042] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:38:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 19:38:12.657217 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8168986-21e1-49ce-b26b-007b6e739009 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:12.658042 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1611f95-3d19-41b9-93dc-c433732ad5b8 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:12.663680 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024c19f8-72e7-4045-ab65-6d8970d9d07e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52a0fce231ba4785a685ac9898d07503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:12.664328 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024c19f8-72e7-4045-ab65-6d8970d9d07e service placement] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:12.664614 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024c19f8-72e7-4045-ab65-6d8970d9d07e service placement] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:12.664896 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024c19f8-72e7-4045-ab65-6d8970d9d07e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:12.667569 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8168986-21e1-49ce-b26b-007b6e739009 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:12.672435 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656eef2a-6b16-4c52-b07c-8adfb1301093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:12.673079 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656eef2a-6b16-4c52-b07c-8adfb1301093 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:12.673423 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656eef2a-6b16-4c52-b07c-8adfb1301093 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:12.673522 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024c19f8-72e7-4045-ab65-6d8970d9d07e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:12.673703 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656eef2a-6b16-4c52-b07c-8adfb1301093 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:12.674485 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656eef2a-6b16-4c52-b07c-8adfb1301093 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:12.675039 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024c19f8-72e7-4045-ab65-6d8970d9d07e service placement] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:12.676658 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656eef2a-6b16-4c52-b07c-8adfb1301093 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:12.685448 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1026/2043] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:38:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 19:38:12.688585 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1018/2044] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:38:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 19:38:12.692712 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024c19f8-72e7-4045-ab65-6d8970d9d07e service placement] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:12.702823 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024c19f8-72e7-4045-ab65-6d8970d9d07e service placement] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:12.707569 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8d21ed-9d72-4d3a-9660-63febc3fd300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:12.708890 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8d21ed-9d72-4d3a-9660-63febc3fd300 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:12.708890 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8d21ed-9d72-4d3a-9660-63febc3fd300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:12.709070 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8d21ed-9d72-4d3a-9660-63febc3fd300 admin admin] PATH_INFO: `/v3/projects/2cf61a9e500e4727aa3406769cc160bd/users/18725150afc14a48a78adad242cf7390/roles/28763c6e70094116b3a4b2895895fa58` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:12.724700 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8d21ed-9d72-4d3a-9660-63febc3fd300 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cf61a9e500e4727aa3406769cc160bd, user_id=18725150afc14a48a78adad242cf7390, role_id=28763c6e70094116b3a4b2895895fa58)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:12.727428 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8d21ed-9d72-4d3a-9660-63febc3fd300 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:12.750285 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656eef2a-6b16-4c52-b07c-8adfb1301093 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:12.752010 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-656eef2a-6b16-4c52-b07c-8adfb1301093 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:12.765146 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656eef2a-6b16-4c52-b07c-8adfb1301093 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:12.769925 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f75c625-c9b5-4a7d-9404-687c4cf205c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:12.770802 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f75c625-c9b5-4a7d-9404-687c4cf205c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:12.771082 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f75c625-c9b5-4a7d-9404-687c4cf205c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:12.771359 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f75c625-c9b5-4a7d-9404-687c4cf205c1 admin admin] PATH_INFO: `/v3/users/3d2ee510f0a14305ad13f85d2e0b3a49` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:12.772553 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f75c625-c9b5-4a7d-9404-687c4cf205c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d2ee510f0a14305ad13f85d2e0b3a49)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:12.774557 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f75c625-c9b5-4a7d-9404-687c4cf205c1 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:12.854653 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1027/2045] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:38:12 2022] PUT /identity/v3/projects/2cf61a9e500e4727aa3406769cc160bd/users/18725150afc14a48a78adad242cf7390/roles/28763c6e70094116b3a4b2895895fa58 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:38:12.872244 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cb4166-6655-4831-9225-34cbec6df9a7 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:12.872540 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cb4166-6655-4831-9225-34cbec6df9a7 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:12.872823 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cb4166-6655-4831-9225-34cbec6df9a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:12.898592 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-87cb4166-6655-4831-9225-34cbec6df9a7 None None] MFA Rules not processed for user `18725150afc14a48a78adad242cf7390`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:38:12.899785 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-87cb4166-6655-4831-9225-34cbec6df9a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:12.962763 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1028/2046] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:38:12 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:38:12.967154 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cb4166-6655-4831-9225-34cbec6df9a7 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:13.029434 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9f75c625-c9b5-4a7d-9404-687c4cf205c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d2ee510f0a14305ad13f85d2e0b3a49'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:13.037275 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cb4166-6655-4831-9225-34cbec6df9a7 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:13.043035 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638bd32f-3f7f-4200-9e66-283b56ed0bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:13.047687 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638bd32f-3f7f-4200-9e66-283b56ed0bc1 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:13.047687 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638bd32f-3f7f-4200-9e66-283b56ed0bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:13.047920 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9f75c625-c9b5-4a7d-9404-687c4cf205c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d2ee510f0a14305ad13f85d2e0b3a49'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:13.048029 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638bd32f-3f7f-4200-9e66-283b56ed0bc1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:13.048029 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638bd32f-3f7f-4200-9e66-283b56ed0bc1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:13.048356 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638bd32f-3f7f-4200-9e66-283b56ed0bc1 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:13.162132 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9f75c625-c9b5-4a7d-9404-687c4cf205c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d2ee510f0a14305ad13f85d2e0b3a49'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:13.162783 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9f75c625-c9b5-4a7d-9404-687c4cf205c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d2ee510f0a14305ad13f85d2e0b3a49'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:13.170187 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9f75c625-c9b5-4a7d-9404-687c4cf205c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d2ee510f0a14305ad13f85d2e0b3a49'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:13.176176 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1019/2047] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jul 15 19:38:12 2022] DELETE /identity/v3/users/3d2ee510f0a14305ad13f85d2e0b3a49 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:38:13.211800 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1029/2048] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jul 15 19:38:12 2022] POST /identity/v3/projects => generated 367 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:38:13.217538 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f75c625-c9b5-4a7d-9404-687c4cf205c1 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:13.219056 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9f75c625-c9b5-4a7d-9404-687c4cf205c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:13.247532 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:38:13.300628 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f75c625-c9b5-4a7d-9404-687c4cf205c1 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:13.371324 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d8cd33-2bb0-4ea1-8f33-d739ba6c9f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:13.372064 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d8cd33-2bb0-4ea1-8f33-d739ba6c9f58 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:13.372348 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d8cd33-2bb0-4ea1-8f33-d739ba6c9f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:13.372631 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d8cd33-2bb0-4ea1-8f33-d739ba6c9f58 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:13.373328 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d8cd33-2bb0-4ea1-8f33-d739ba6c9f58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:13.375362 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d8cd33-2bb0-4ea1-8f33-d739ba6c9f58 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:13.531052 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1020/2049] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jul 15 19:38:13 2022] POST /identity/v3/users => generated 290 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:38:13.535106 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638bd32f-3f7f-4200-9e66-283b56ed0bc1 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:13.545257 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638bd32f-3f7f-4200-9e66-283b56ed0bc1 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:13.549704 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80007568-4d40-4462-adb8-e959c4f7ebd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:13.550427 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80007568-4d40-4462-adb8-e959c4f7ebd3 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:13.550759 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80007568-4d40-4462-adb8-e959c4f7ebd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:13.550985 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80007568-4d40-4462-adb8-e959c4f7ebd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:13.551705 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80007568-4d40-4462-adb8-e959c4f7ebd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:13.553283 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80007568-4d40-4462-adb8-e959c4f7ebd3 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:13.563732 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1030/2050] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:38:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:38:13.568347 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d8cd33-2bb0-4ea1-8f33-d739ba6c9f58 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:13.578825 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d8cd33-2bb0-4ea1-8f33-d739ba6c9f58 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:13.584154 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6f4153-9444-454d-98d0-80a070deead7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:13.585143 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6f4153-9444-454d-98d0-80a070deead7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:13.585432 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6f4153-9444-454d-98d0-80a070deead7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:13.585707 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6f4153-9444-454d-98d0-80a070deead7 admin admin] PATH_INFO: `/v3/projects/7d095c68b59142cdb356118a6593bb31/users/de617841c2df451a88736d6ef441abf2/roles/4318f86ba9f74cd4a0bef1c311276657` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:13.601833 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6f4153-9444-454d-98d0-80a070deead7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d095c68b59142cdb356118a6593bb31, user_id=de617841c2df451a88736d6ef441abf2, role_id=4318f86ba9f74cd4a0bef1c311276657)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:13.604764 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6f4153-9444-454d-98d0-80a070deead7 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:13.681550 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80007568-4d40-4462-adb8-e959c4f7ebd3 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:13.683363 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-80007568-4d40-4462-adb8-e959c4f7ebd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:13.697679 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80007568-4d40-4462-adb8-e959c4f7ebd3 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:13.703104 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dc142d-60bb-49d5-8d0b-121b025daff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:13.704051 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dc142d-60bb-49d5-8d0b-121b025daff9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:13.704272 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dc142d-60bb-49d5-8d0b-121b025daff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:13.704671 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dc142d-60bb-49d5-8d0b-121b025daff9 admin admin] PATH_INFO: `/v3/projects/7797a303c4d24e988742fcbebbfa6baa` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:13.706074 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dc142d-60bb-49d5-8d0b-121b025daff9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7797a303c4d24e988742fcbebbfa6baa)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:13.708623 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dc142d-60bb-49d5-8d0b-121b025daff9 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:13.726799 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:38:13.774969 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1021/2051] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:38:13 2022] PUT /identity/v3/projects/7d095c68b59142cdb356118a6593bb31/users/de617841c2df451a88736d6ef441abf2/roles/4318f86ba9f74cd4a0bef1c311276657 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:38:13.779290 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6f4153-9444-454d-98d0-80a070deead7 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:13.843809 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6f4153-9444-454d-98d0-80a070deead7 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:13.850134 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5511299e-1086-4a8a-9e43-6665d4a400c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:13.850815 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5511299e-1086-4a8a-9e43-6665d4a400c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:13.851096 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5511299e-1086-4a8a-9e43-6665d4a400c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:13.851378 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5511299e-1086-4a8a-9e43-6665d4a400c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:13.852109 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5511299e-1086-4a8a-9e43-6665d4a400c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:13.853729 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5511299e-1086-4a8a-9e43-6665d4a400c0 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:13.868887 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1022/2052] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:38:13 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:38:13.869059 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-15dc142d-60bb-49d5-8d0b-121b025daff9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7797a303c4d24e988742fcbebbfa6baa', 'request_id': 'req-15dc142d-60bb-49d5-8d0b-121b025daff9'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:13.870898 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5511299e-1086-4a8a-9e43-6665d4a400c0 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:13.882313 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5511299e-1086-4a8a-9e43-6665d4a400c0 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:13.903146 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:38:13.943267 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4766d0-08c7-4776-a925-a9ec9105a4af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:13.944172 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4766d0-08c7-4776-a925-a9ec9105a4af admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:13.944456 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4766d0-08c7-4776-a925-a9ec9105a4af admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:13.944732 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4766d0-08c7-4776-a925-a9ec9105a4af admin admin] PATH_INFO: `/v3/projects/7d095c68b59142cdb356118a6593bb31/users/de617841c2df451a88736d6ef441abf2/roles/4318f86ba9f74cd4a0bef1c311276657` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:13.946800 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4766d0-08c7-4776-a925-a9ec9105a4af admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d095c68b59142cdb356118a6593bb31, user_id=de617841c2df451a88736d6ef441abf2, role_id=4318f86ba9f74cd4a0bef1c311276657)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:13.949463 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4766d0-08c7-4776-a925-a9ec9105a4af admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:13.960505 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1023/2053] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:38:13 2022] PUT /identity/v3/projects/7d095c68b59142cdb356118a6593bb31/users/de617841c2df451a88736d6ef441abf2/roles/4318f86ba9f74cd4a0bef1c311276657 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:38:13.972812 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300f1b5f-f87b-41d9-9715-2914e761f76e None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:13.973252 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300f1b5f-f87b-41d9-9715-2914e761f76e None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:13.973527 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300f1b5f-f87b-41d9-9715-2914e761f76e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:13.985074 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-15dc142d-60bb-49d5-8d0b-121b025daff9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7797a303c4d24e988742fcbebbfa6baa', 'request_id': 'req-15dc142d-60bb-49d5-8d0b-121b025daff9'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:13.992683 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-15dc142d-60bb-49d5-8d0b-121b025daff9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7797a303c4d24e988742fcbebbfa6baa', 'request_id': 'req-15dc142d-60bb-49d5-8d0b-121b025daff9'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:14.001503 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-300f1b5f-f87b-41d9-9715-2914e761f76e None None] MFA Rules not processed for user `de617841c2df451a88736d6ef441abf2`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:38:14.002964 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-300f1b5f-f87b-41d9-9715-2914e761f76e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:14.003791 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-15dc142d-60bb-49d5-8d0b-121b025daff9 admin admin] The token cache is being invalidate because project 7797a303c4d24e988742fcbebbfa6baa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:38:14.004468 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-15dc142d-60bb-49d5-8d0b-121b025daff9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:14.005579 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-15dc142d-60bb-49d5-8d0b-121b025daff9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:14.007971 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1031/2054] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jul 15 19:38:13 2022] DELETE /identity/v3/projects/7797a303c4d24e988742fcbebbfa6baa => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:38:14.015032 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d4c7ff-0da9-4d1c-975b-a8a9a1858ec4 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:14.015402 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d4c7ff-0da9-4d1c-975b-a8a9a1858ec4 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:14.015760 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d4c7ff-0da9-4d1c-975b-a8a9a1858ec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:14.039838 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-c1d4c7ff-0da9-4d1c-975b-a8a9a1858ec4 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:38:14.041321 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-c1d4c7ff-0da9-4d1c-975b-a8a9a1858ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:14.095306 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1024/2055] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:38:13 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:38:14.109785 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300f1b5f-f87b-41d9-9715-2914e761f76e None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:14.111277 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-300f1b5f-f87b-41d9-9715-2914e761f76e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:14.114466 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1032/2056] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:38:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:38:14.122265 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f06bc7-f59f-42d7-bb99-7bcaff0120d3 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:14.122563 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f06bc7-f59f-42d7-bb99-7bcaff0120d3 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:14.122987 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f06bc7-f59f-42d7-bb99-7bcaff0120d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:14.144298 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-89f06bc7-f59f-42d7-bb99-7bcaff0120d3 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:38:14.145663 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-89f06bc7-f59f-42d7-bb99-7bcaff0120d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:14.154895 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1033/2057] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:38:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:38:14.162124 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f06bc7-f59f-42d7-bb99-7bcaff0120d3 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:14.172300 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f06bc7-f59f-42d7-bb99-7bcaff0120d3 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:14.177288 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a2b36b-4156-48cc-9060-690984d4735e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:14.178002 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a2b36b-4156-48cc-9060-690984d4735e admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:14.178287 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a2b36b-4156-48cc-9060-690984d4735e admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:14.178566 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a2b36b-4156-48cc-9060-690984d4735e admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:14.179299 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a2b36b-4156-48cc-9060-690984d4735e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:38:14.179602 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a2b36b-4156-48cc-9060-690984d4735e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:14.181245 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a2b36b-4156-48cc-9060-690984d4735e admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:14.195575 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1034/2058] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jul 15 19:38:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:38:14.200300 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a2b36b-4156-48cc-9060-690984d4735e admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:14.203035 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300f1b5f-f87b-41d9-9715-2914e761f76e None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:14.213440 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3bfa0a-2af5-43e5-ae9c-619f26b4261c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86c2df3d378490ca7ea497a73c19d40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:14.214391 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3bfa0a-2af5-43e5-ae9c-619f26b4261c service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:14.214624 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a2b36b-4156-48cc-9060-690984d4735e admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:14.214992 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3bfa0a-2af5-43e5-ae9c-619f26b4261c service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:14.215524 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3bfa0a-2af5-43e5-ae9c-619f26b4261c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:14.220825 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30d27b4-f062-4247-8eb9-57b46db0a92f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:14.221791 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d27b4-f062-4247-8eb9-57b46db0a92f admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:14.221791 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d27b4-f062-4247-8eb9-57b46db0a92f admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:14.222083 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d27b4-f062-4247-8eb9-57b46db0a92f admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:14.223138 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30d27b4-f062-4247-8eb9-57b46db0a92f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:14.225229 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30d27b4-f062-4247-8eb9-57b46db0a92f admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:14.228029 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3bfa0a-2af5-43e5-ae9c-619f26b4261c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:14.229528 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3bfa0a-2af5-43e5-ae9c-619f26b4261c service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:14.241563 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1025/2059] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:38:14 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 19:38:14.398924 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1035/2060] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jul 15 19:38:14 2022] POST /identity/v3/projects => generated 365 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:38:14.399071 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3bfa0a-2af5-43e5-ae9c-619f26b4261c service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:14.412768 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3bfa0a-2af5-43e5-ae9c-619f26b4261c service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:14.468002 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:38:14.493917 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba07adcf-6696-4599-8507-2ccfa5d30846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:14.494740 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba07adcf-6696-4599-8507-2ccfa5d30846 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:14.495023 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba07adcf-6696-4599-8507-2ccfa5d30846 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:14.495297 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba07adcf-6696-4599-8507-2ccfa5d30846 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:14.496060 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba07adcf-6696-4599-8507-2ccfa5d30846 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:14.498199 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba07adcf-6696-4599-8507-2ccfa5d30846 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:14.612345 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30d27b4-f062-4247-8eb9-57b46db0a92f admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:14.650456 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1026/2061] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jul 15 19:38:14 2022] POST /identity/v3/users => generated 283 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:38:14.651579 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba07adcf-6696-4599-8507-2ccfa5d30846 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:14.661823 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba07adcf-6696-4599-8507-2ccfa5d30846 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:14.666546 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6c4261-cfff-46f7-b05c-0d04abada254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:14.667209 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6c4261-cfff-46f7-b05c-0d04abada254 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:14.667492 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6c4261-cfff-46f7-b05c-0d04abada254 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:14.667771 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6c4261-cfff-46f7-b05c-0d04abada254 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:14.668503 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6c4261-cfff-46f7-b05c-0d04abada254 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:14.670168 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6c4261-cfff-46f7-b05c-0d04abada254 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:14.680813 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1027/2062] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:38:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:38:14.684681 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6c4261-cfff-46f7-b05c-0d04abada254 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:14.688605 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30d27b4-f062-4247-8eb9-57b46db0a92f admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:14.692467 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303bd1ef-1c1d-4389-a97e-8a63c0d95cf2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86c2df3d378490ca7ea497a73c19d40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:14.693127 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303bd1ef-1c1d-4389-a97e-8a63c0d95cf2 service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:14.693530 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303bd1ef-1c1d-4389-a97e-8a63c0d95cf2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:14.693840 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303bd1ef-1c1d-4389-a97e-8a63c0d95cf2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:14.694591 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6c4261-cfff-46f7-b05c-0d04abada254 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:14.695709 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-303bd1ef-1c1d-4389-a97e-8a63c0d95cf2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:14.699360 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0c51fc-0b8c-4d78-8647-db19758def27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:14.700078 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0c51fc-0b8c-4d78-8647-db19758def27 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:14.700364 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0c51fc-0b8c-4d78-8647-db19758def27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:14.700656 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0c51fc-0b8c-4d78-8647-db19758def27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:14.701367 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0c51fc-0b8c-4d78-8647-db19758def27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:14.702990 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0c51fc-0b8c-4d78-8647-db19758def27 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:14.714012 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1028/2063] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:38:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 19:38:14.716337 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:38:14.718256 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0c51fc-0b8c-4d78-8647-db19758def27 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:14.729193 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0c51fc-0b8c-4d78-8647-db19758def27 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:14.733978 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa808a5-a02e-4681-a961-7debe1c8fba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:14.734907 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa808a5-a02e-4681-a961-7debe1c8fba4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:14.735188 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa808a5-a02e-4681-a961-7debe1c8fba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:14.735462 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa808a5-a02e-4681-a961-7debe1c8fba4 admin admin] PATH_INFO: `/v3/projects/55a95e8c28764b989f0b454cbac4c30c/users/7be011e18a384decac9c016d4251a200/roles/28763c6e70094116b3a4b2895895fa58` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:14.749974 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa808a5-a02e-4681-a961-7debe1c8fba4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55a95e8c28764b989f0b454cbac4c30c, user_id=7be011e18a384decac9c016d4251a200, role_id=28763c6e70094116b3a4b2895895fa58)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:14.752681 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa808a5-a02e-4681-a961-7debe1c8fba4 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:14.755316 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303bd1ef-1c1d-4389-a97e-8a63c0d95cf2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:14.756840 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303bd1ef-1c1d-4389-a97e-8a63c0d95cf2 service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:14.765935 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1036/2064] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:38:14 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 19:38:14.919818 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1029/2065] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:38:14 2022] PUT /identity/v3/projects/55a95e8c28764b989f0b454cbac4c30c/users/7be011e18a384decac9c016d4251a200/roles/28763c6e70094116b3a4b2895895fa58 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:38:14.930030 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a339887-c26d-4986-925f-687c1be55b7b None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:14.930327 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a339887-c26d-4986-925f-687c1be55b7b None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:14.930621 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a339887-c26d-4986-925f-687c1be55b7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:14.959953 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-2a339887-c26d-4986-925f-687c1be55b7b None None] MFA Rules not processed for user `7be011e18a384decac9c016d4251a200`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:38:14.969154 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-2a339887-c26d-4986-925f-687c1be55b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:15.038595 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1037/2066] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:38:14 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:38:15.049959 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa808a5-a02e-4681-a961-7debe1c8fba4 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:15.198919 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-efa808a5-a02e-4681-a961-7debe1c8fba4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:15.198919 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa808a5-a02e-4681-a961-7debe1c8fba4 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:15.198919 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edecd53-ff5e-4c60-ba20-b3cbaf73807d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86c2df3d378490ca7ea497a73c19d40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:15.200536 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edecd53-ff5e-4c60-ba20-b3cbaf73807d service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:15.200536 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edecd53-ff5e-4c60-ba20-b3cbaf73807d service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:15.200536 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edecd53-ff5e-4c60-ba20-b3cbaf73807d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:15.200536 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edecd53-ff5e-4c60-ba20-b3cbaf73807d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:15.200536 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edecd53-ff5e-4c60-ba20-b3cbaf73807d service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:15.200536 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1030/2067] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:38:15 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:38:18.975978 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c283fe03-f249-4daa-aa9a-b58db5de7f29 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:18.976451 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c283fe03-f249-4daa-aa9a-b58db5de7f29 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:18.976557 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c283fe03-f249-4daa-aa9a-b58db5de7f29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:18.994288 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-c283fe03-f249-4daa-aa9a-b58db5de7f29 None None] MFA Rules not processed for user `3d893c14620b4a129842062f14a68a78`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:38:18.997742 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-c283fe03-f249-4daa-aa9a-b58db5de7f29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:19.061236 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1038/2068] 10.222.0.33 () {58 vars in 1009 bytes} [Fri Jul 15 19:38:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:38:19.103286 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edecd53-ff5e-4c60-ba20-b3cbaf73807d service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:19.104742 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-5edecd53-ff5e-4c60-ba20-b3cbaf73807d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:19.182952 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edecd53-ff5e-4c60-ba20-b3cbaf73807d service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:19.188520 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0b72e0-d271-4c89-8c1a-63233f8c37a5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef76f7fe39a3499aaeebf85c176804f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:19.189222 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b72e0-d271-4c89-8c1a-63233f8c37a5 service swift] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:19.189498 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b72e0-d271-4c89-8c1a-63233f8c37a5 service swift] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:19.189787 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b72e0-d271-4c89-8c1a-63233f8c37a5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:19.199699 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0b72e0-d271-4c89-8c1a-63233f8c37a5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:19.201491 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0b72e0-d271-4c89-8c1a-63233f8c37a5 service swift] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:19.212277 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1031/2069] 10.222.0.33 () {60 vars in 1384 bytes} [Fri Jul 15 19:38:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 19:38:20.336310 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1793cd69-86b1-465b-bf3e-7033a2e5ab00 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:20.336697 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1793cd69-86b1-465b-bf3e-7033a2e5ab00 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:20.336879 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1793cd69-86b1-465b-bf3e-7033a2e5ab00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:20.392235 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-1793cd69-86b1-465b-bf3e-7033a2e5ab00 None None] MFA Rules not processed for user `3d893c14620b4a129842062f14a68a78`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:38:20.392235 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-1793cd69-86b1-465b-bf3e-7033a2e5ab00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:20.392235 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1039/2070] 10.222.0.33 () {58 vars in 967 bytes} [Fri Jul 15 19:38:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:38:20.393926 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0b72e0-d271-4c89-8c1a-63233f8c37a5 service swift] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:20.394962 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0b72e0-d271-4c89-8c1a-63233f8c37a5 service swift] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:20.403807 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd39d916-7e48-4cb8-af52-9f2c36f15104 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef76f7fe39a3499aaeebf85c176804f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:20.404621 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd39d916-7e48-4cb8-af52-9f2c36f15104 service swift] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:20.405039 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd39d916-7e48-4cb8-af52-9f2c36f15104 service swift] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:20.405454 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd39d916-7e48-4cb8-af52-9f2c36f15104 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:20.415399 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd39d916-7e48-4cb8-af52-9f2c36f15104 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:20.417211 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd39d916-7e48-4cb8-af52-9f2c36f15104 service swift] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:20.426530 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1032/2071] 10.222.0.33 () {60 vars in 1384 bytes} [Fri Jul 15 19:38:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 19:38:21.383072 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bcc829-5266-4119-bed2-76d7c9b5dc2a None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:21.383907 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bcc829-5266-4119-bed2-76d7c9b5dc2a None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:21.384186 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bcc829-5266-4119-bed2-76d7c9b5dc2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:21.402328 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-e9bcc829-5266-4119-bed2-76d7c9b5dc2a None None] MFA Rules not processed for user `3d893c14620b4a129842062f14a68a78`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:38:21.403545 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-e9bcc829-5266-4119-bed2-76d7c9b5dc2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:21.411648 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1040/2072] 10.222.0.33 () {58 vars in 1009 bytes} [Fri Jul 15 19:38:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:38:21.423555 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd39d916-7e48-4cb8-af52-9f2c36f15104 service swift] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:21.433889 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd39d916-7e48-4cb8-af52-9f2c36f15104 service swift] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:21.438943 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f450d268-ec83-480b-903e-acbe88f2da82 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef76f7fe39a3499aaeebf85c176804f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:21.439585 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f450d268-ec83-480b-903e-acbe88f2da82 service swift] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:21.439870 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f450d268-ec83-480b-903e-acbe88f2da82 service swift] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:21.440143 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f450d268-ec83-480b-903e-acbe88f2da82 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:21.450476 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:38:21.453159 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f450d268-ec83-480b-903e-acbe88f2da82 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:21.456752 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f450d268-ec83-480b-903e-acbe88f2da82 service swift] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:21.469049 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1033/2073] 10.222.0.33 () {60 vars in 1384 bytes} [Fri Jul 15 19:38:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 19:38:22.271997 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae34ed0-dc12-4ff6-827f-dd394f4b5acf None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:22.272397 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae34ed0-dc12-4ff6-827f-dd394f4b5acf None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:22.272745 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae34ed0-dc12-4ff6-827f-dd394f4b5acf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:22.291575 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-9ae34ed0-dc12-4ff6-827f-dd394f4b5acf None None] MFA Rules not processed for user `3d893c14620b4a129842062f14a68a78`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:38:22.292899 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-9ae34ed0-dc12-4ff6-827f-dd394f4b5acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:22.300449 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1041/2074] 10.222.0.33 () {58 vars in 967 bytes} [Fri Jul 15 19:38:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:38:22.312871 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f450d268-ec83-480b-903e-acbe88f2da82 service swift] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:22.327345 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f450d268-ec83-480b-903e-acbe88f2da82 service swift] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:22.333185 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e99f42-9411-4700-bcfe-111c54d32010 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef76f7fe39a3499aaeebf85c176804f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:22.333863 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e99f42-9411-4700-bcfe-111c54d32010 service swift] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:22.334288 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e99f42-9411-4700-bcfe-111c54d32010 service swift] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:22.334569 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e99f42-9411-4700-bcfe-111c54d32010 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:22.343162 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e99f42-9411-4700-bcfe-111c54d32010 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:22.344753 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e99f42-9411-4700-bcfe-111c54d32010 service swift] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:22.354129 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1034/2075] 10.222.0.33 () {60 vars in 1384 bytes} [Fri Jul 15 19:38:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 19:38:24.133678 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae34ed0-dc12-4ff6-827f-dd394f4b5acf None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:24.135726 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-9ae34ed0-dc12-4ff6-827f-dd394f4b5acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:24.194479 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:38:24.229483 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae34ed0-dc12-4ff6-827f-dd394f4b5acf None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:24.234421 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcd89cc-0ca7-4144-b137-38861e22c684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:24.235561 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcd89cc-0ca7-4144-b137-38861e22c684 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:24.236091 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcd89cc-0ca7-4144-b137-38861e22c684 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:24.236310 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcd89cc-0ca7-4144-b137-38861e22c684 admin admin] PATH_INFO: `/v3/users/18725150afc14a48a78adad242cf7390` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:24.238160 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcd89cc-0ca7-4144-b137-38861e22c684 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18725150afc14a48a78adad242cf7390)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:24.242205 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcd89cc-0ca7-4144-b137-38861e22c684 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:24.515521 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-8fcd89cc-0ca7-4144-b137-38861e22c684 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18725150afc14a48a78adad242cf7390'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:24.530656 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-8fcd89cc-0ca7-4144-b137-38861e22c684 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18725150afc14a48a78adad242cf7390'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:24.652663 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-8fcd89cc-0ca7-4144-b137-38861e22c684 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18725150afc14a48a78adad242cf7390'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:24.653306 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-8fcd89cc-0ca7-4144-b137-38861e22c684 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18725150afc14a48a78adad242cf7390'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:24.653969 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-8fcd89cc-0ca7-4144-b137-38861e22c684 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18725150afc14a48a78adad242cf7390'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:24.668508 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1042/2076] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jul 15 19:38:24 2022] DELETE /identity/v3/users/18725150afc14a48a78adad242cf7390 => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:38:24.673027 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc1fdf2-fc91-4100-ac4a-f2d1c5f43c82 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:24.677208 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc1fdf2-fc91-4100-ac4a-f2d1c5f43c82 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:24.677208 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc1fdf2-fc91-4100-ac4a-f2d1c5f43c82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:24.691931 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-2bc1fdf2-fc91-4100-ac4a-f2d1c5f43c82 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:38:24.693209 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-2bc1fdf2-fc91-4100-ac4a-f2d1c5f43c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:24.751419 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1035/2077] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:38:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:38:24.762525 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcd89cc-0ca7-4144-b137-38861e22c684 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:24.763972 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-8fcd89cc-0ca7-4144-b137-38861e22c684 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:24.825354 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcd89cc-0ca7-4144-b137-38861e22c684 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:24.828914 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9d8b40-e47c-4103-96fb-006805e2cb11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:24.829556 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d8b40-e47c-4103-96fb-006805e2cb11 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:24.829869 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d8b40-e47c-4103-96fb-006805e2cb11 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:24.830147 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d8b40-e47c-4103-96fb-006805e2cb11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:24.839926 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9d8b40-e47c-4103-96fb-006805e2cb11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:24.841363 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9d8b40-e47c-4103-96fb-006805e2cb11 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:24.851227 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1043/2078] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:38:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:38:25.282873 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc1fdf2-fc91-4100-ac4a-f2d1c5f43c82 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:25.285206 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-2bc1fdf2-fc91-4100-ac4a-f2d1c5f43c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:25.297905 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc1fdf2-fc91-4100-ac4a-f2d1c5f43c82 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:25.302846 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101fb24b-7563-4fe1-bd05-5cecf0b62934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:25.303721 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101fb24b-7563-4fe1-bd05-5cecf0b62934 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:25.304004 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101fb24b-7563-4fe1-bd05-5cecf0b62934 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:25.304286 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101fb24b-7563-4fe1-bd05-5cecf0b62934 admin admin] PATH_INFO: `/v3/projects/2cf61a9e500e4727aa3406769cc160bd` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:25.305493 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101fb24b-7563-4fe1-bd05-5cecf0b62934 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2cf61a9e500e4727aa3406769cc160bd)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:25.307463 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101fb24b-7563-4fe1-bd05-5cecf0b62934 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:25.429308 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-101fb24b-7563-4fe1-bd05-5cecf0b62934 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2cf61a9e500e4727aa3406769cc160bd', 'request_id': 'req-101fb24b-7563-4fe1-bd05-5cecf0b62934'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:25.599015 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-101fb24b-7563-4fe1-bd05-5cecf0b62934 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2cf61a9e500e4727aa3406769cc160bd', 'request_id': 'req-101fb24b-7563-4fe1-bd05-5cecf0b62934'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:25.605143 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-101fb24b-7563-4fe1-bd05-5cecf0b62934 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2cf61a9e500e4727aa3406769cc160bd', 'request_id': 'req-101fb24b-7563-4fe1-bd05-5cecf0b62934'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:25.616554 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-101fb24b-7563-4fe1-bd05-5cecf0b62934 admin admin] The token cache is being invalidate because project 2cf61a9e500e4727aa3406769cc160bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:38:25.616844 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-101fb24b-7563-4fe1-bd05-5cecf0b62934 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:25.617474 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-101fb24b-7563-4fe1-bd05-5cecf0b62934 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:25.619053 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1036/2079] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jul 15 19:38:25 2022] DELETE /identity/v3/projects/2cf61a9e500e4727aa3406769cc160bd => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:38:25.622967 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9d8b40-e47c-4103-96fb-006805e2cb11 service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:25.633688 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9d8b40-e47c-4103-96fb-006805e2cb11 service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:25.685788 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:38:25.698210 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b264fa0-5181-4905-87d4-f64b37bf86ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:25.698210 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b264fa0-5181-4905-87d4-f64b37bf86ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:25.698210 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b264fa0-5181-4905-87d4-f64b37bf86ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:25.698210 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b264fa0-5181-4905-87d4-f64b37bf86ad admin admin] PATH_INFO: `/v3/users/de617841c2df451a88736d6ef441abf2` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:25.698778 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b264fa0-5181-4905-87d4-f64b37bf86ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=de617841c2df451a88736d6ef441abf2)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:25.698778 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b264fa0-5181-4905-87d4-f64b37bf86ad admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:25.929320 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-6b264fa0-5181-4905-87d4-f64b37bf86ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de617841c2df451a88736d6ef441abf2'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:25.944712 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-6b264fa0-5181-4905-87d4-f64b37bf86ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de617841c2df451a88736d6ef441abf2'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:26.021554 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-6b264fa0-5181-4905-87d4-f64b37bf86ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de617841c2df451a88736d6ef441abf2'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:26.022211 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-6b264fa0-5181-4905-87d4-f64b37bf86ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de617841c2df451a88736d6ef441abf2'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:26.022815 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-6b264fa0-5181-4905-87d4-f64b37bf86ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de617841c2df451a88736d6ef441abf2'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:26.035616 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1044/2080] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jul 15 19:38:25 2022] DELETE /identity/v3/users/de617841c2df451a88736d6ef441abf2 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:38:26.592865 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101fb24b-7563-4fe1-bd05-5cecf0b62934 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:26.594340 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-101fb24b-7563-4fe1-bd05-5cecf0b62934 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:26.617635 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101fb24b-7563-4fe1-bd05-5cecf0b62934 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:26.692720 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d58915a-a4e8-493d-ac3b-85156d2ce8d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:26.693596 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d58915a-a4e8-493d-ac3b-85156d2ce8d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:26.694070 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d58915a-a4e8-493d-ac3b-85156d2ce8d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:26.694243 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d58915a-a4e8-493d-ac3b-85156d2ce8d0 admin admin] PATH_INFO: `/v3/projects/7d095c68b59142cdb356118a6593bb31` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:26.695586 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d58915a-a4e8-493d-ac3b-85156d2ce8d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d095c68b59142cdb356118a6593bb31)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:26.697501 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d58915a-a4e8-493d-ac3b-85156d2ce8d0 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:26.704787 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:38:26.726205 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b264fa0-5181-4905-87d4-f64b37bf86ad admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:26.728029 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-6b264fa0-5181-4905-87d4-f64b37bf86ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:26.740895 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b264fa0-5181-4905-87d4-f64b37bf86ad admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:26.747989 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef822ca-44cb-434d-a426-377f83ac5a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:26.753165 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef822ca-44cb-434d-a426-377f83ac5a18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:26.753165 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef822ca-44cb-434d-a426-377f83ac5a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:26.753165 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef822ca-44cb-434d-a426-377f83ac5a18 admin admin] PATH_INFO: `/v3/users/816412dd69984ffa949e2129312b1379` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:26.753165 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef822ca-44cb-434d-a426-377f83ac5a18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=816412dd69984ffa949e2129312b1379)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:26.753165 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef822ca-44cb-434d-a426-377f83ac5a18 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:26.841608 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9d58915a-a4e8-493d-ac3b-85156d2ce8d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d095c68b59142cdb356118a6593bb31', 'request_id': 'req-9d58915a-a4e8-493d-ac3b-85156d2ce8d0'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:26.938380 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9d58915a-a4e8-493d-ac3b-85156d2ce8d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d095c68b59142cdb356118a6593bb31', 'request_id': 'req-9d58915a-a4e8-493d-ac3b-85156d2ce8d0'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:26.945196 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9d58915a-a4e8-493d-ac3b-85156d2ce8d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d095c68b59142cdb356118a6593bb31', 'request_id': 'req-9d58915a-a4e8-493d-ac3b-85156d2ce8d0'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:26.956976 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9d58915a-a4e8-493d-ac3b-85156d2ce8d0 admin admin] The token cache is being invalidate because project 7d095c68b59142cdb356118a6593bb31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:38:26.957262 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9d58915a-a4e8-493d-ac3b-85156d2ce8d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:26.957951 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9d58915a-a4e8-493d-ac3b-85156d2ce8d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:26.959577 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1037/2081] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jul 15 19:38:26 2022] DELETE /identity/v3/projects/7d095c68b59142cdb356118a6593bb31 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:38:26.964270 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c68cd6-b8a8-4e24-b426-95bd3bfbf293 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:26.964567 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c68cd6-b8a8-4e24-b426-95bd3bfbf293 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:26.964851 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c68cd6-b8a8-4e24-b426-95bd3bfbf293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:26.982282 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-58c68cd6-b8a8-4e24-b426-95bd3bfbf293 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:38:26.983511 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-58c68cd6-b8a8-4e24-b426-95bd3bfbf293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:27.042024 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1038/2082] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:38:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:38:27.049672 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfda7a3-14aa-494e-a7e1-b8bf33ae871a None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:27.049997 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfda7a3-14aa-494e-a7e1-b8bf33ae871a None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:27.050278 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfda7a3-14aa-494e-a7e1-b8bf33ae871a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:27.068329 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-0bfda7a3-14aa-494e-a7e1-b8bf33ae871a None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:38:27.069541 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0bfda7a3-14aa-494e-a7e1-b8bf33ae871a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:27.077605 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1039/2083] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:38:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:38:27.082285 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfda7a3-14aa-494e-a7e1-b8bf33ae871a None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:27.091298 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfda7a3-14aa-494e-a7e1-b8bf33ae871a None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:27.100938 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad59f46b-43e3-4d3e-9794-a99117cb87fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:27.100938 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad59f46b-43e3-4d3e-9794-a99117cb87fb admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:27.100938 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad59f46b-43e3-4d3e-9794-a99117cb87fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:27.100938 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad59f46b-43e3-4d3e-9794-a99117cb87fb admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:27.101344 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad59f46b-43e3-4d3e-9794-a99117cb87fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:38:27.101344 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad59f46b-43e3-4d3e-9794-a99117cb87fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:27.101344 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad59f46b-43e3-4d3e-9794-a99117cb87fb admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:27.104332 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-aef822ca-44cb-434d-a426-377f83ac5a18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '816412dd69984ffa949e2129312b1379'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:27.112279 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1040/2084] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jul 15 19:38:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:38:27.116632 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-aef822ca-44cb-434d-a426-377f83ac5a18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '816412dd69984ffa949e2129312b1379'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:27.116792 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad59f46b-43e3-4d3e-9794-a99117cb87fb admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:27.128242 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad59f46b-43e3-4d3e-9794-a99117cb87fb admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:27.133432 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e64286-cdb2-4527-8c5d-d7645752efa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:27.134114 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e64286-cdb2-4527-8c5d-d7645752efa4 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:27.134398 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e64286-cdb2-4527-8c5d-d7645752efa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:27.134722 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e64286-cdb2-4527-8c5d-d7645752efa4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:27.135421 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e64286-cdb2-4527-8c5d-d7645752efa4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:27.137393 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e64286-cdb2-4527-8c5d-d7645752efa4 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:27.198476 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-aef822ca-44cb-434d-a426-377f83ac5a18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '816412dd69984ffa949e2129312b1379'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:27.199155 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-aef822ca-44cb-434d-a426-377f83ac5a18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '816412dd69984ffa949e2129312b1379'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:27.199788 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-aef822ca-44cb-434d-a426-377f83ac5a18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '816412dd69984ffa949e2129312b1379'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:27.214664 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1045/2085] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jul 15 19:38:26 2022] DELETE /identity/v3/users/816412dd69984ffa949e2129312b1379 => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:38:27.220637 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd73e4d3-64ff-46ed-b7ec-d3b1c14cd0e2 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:27.221065 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd73e4d3-64ff-46ed-b7ec-d3b1c14cd0e2 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:27.221475 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd73e4d3-64ff-46ed-b7ec-d3b1c14cd0e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:27.240432 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1041/2086] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jul 15 19:38:27 2022] POST /identity/v3/projects => generated 373 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:38:27.244610 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e64286-cdb2-4527-8c5d-d7645752efa4 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:27.246487 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d5e64286-cdb2-4527-8c5d-d7645752efa4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:27.258054 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-cd73e4d3-64ff-46ed-b7ec-d3b1c14cd0e2 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:38:27.260618 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-cd73e4d3-64ff-46ed-b7ec-d3b1c14cd0e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:27.306241 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:38:27.318852 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e64286-cdb2-4527-8c5d-d7645752efa4 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:27.323668 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1046/2087] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:38:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:38:27.323888 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fb3ce7-92fe-4ed6-9a2d-6ad89eb1468e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:27.324592 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb3ce7-92fe-4ed6-9a2d-6ad89eb1468e admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:27.324873 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb3ce7-92fe-4ed6-9a2d-6ad89eb1468e admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:27.325148 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb3ce7-92fe-4ed6-9a2d-6ad89eb1468e admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:27.325869 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fb3ce7-92fe-4ed6-9a2d-6ad89eb1468e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:27.328071 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fb3ce7-92fe-4ed6-9a2d-6ad89eb1468e admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:27.333321 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd73e4d3-64ff-46ed-b7ec-d3b1c14cd0e2 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:27.334980 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-cd73e4d3-64ff-46ed-b7ec-d3b1c14cd0e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:27.395167 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd73e4d3-64ff-46ed-b7ec-d3b1c14cd0e2 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:27.398962 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882eac41-d258-4049-9ac3-e401b8c505a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:27.399658 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882eac41-d258-4049-9ac3-e401b8c505a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:27.400000 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882eac41-d258-4049-9ac3-e401b8c505a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:27.404941 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882eac41-d258-4049-9ac3-e401b8c505a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:27.410348 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882eac41-d258-4049-9ac3-e401b8c505a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:27.411770 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882eac41-d258-4049-9ac3-e401b8c505a1 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:27.422170 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1047/2088] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:38:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 19:38:27.465452 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1042/2089] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jul 15 19:38:27 2022] POST /identity/v3/users => generated 287 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:38:27.469734 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882eac41-d258-4049-9ac3-e401b8c505a1 service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:27.481168 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882eac41-d258-4049-9ac3-e401b8c505a1 service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:27.486437 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c777af-98e0-4f93-a45a-586b72f32bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:27.487620 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c777af-98e0-4f93-a45a-586b72f32bd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:27.488232 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c777af-98e0-4f93-a45a-586b72f32bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:27.488232 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c777af-98e0-4f93-a45a-586b72f32bd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:27.488878 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c777af-98e0-4f93-a45a-586b72f32bd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:27.490524 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c777af-98e0-4f93-a45a-586b72f32bd9 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:27.501040 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1048/2090] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:38:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 19:38:27.507016 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fb3ce7-92fe-4ed6-9a2d-6ad89eb1468e admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:27.523835 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fb3ce7-92fe-4ed6-9a2d-6ad89eb1468e admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:27.530504 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976e618b-197c-4d92-89d6-15aec0d48ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:27.530504 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976e618b-197c-4d92-89d6-15aec0d48ab5 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:27.530504 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976e618b-197c-4d92-89d6-15aec0d48ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:27.531121 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976e618b-197c-4d92-89d6-15aec0d48ab5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:27.531327 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976e618b-197c-4d92-89d6-15aec0d48ab5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:27.532998 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976e618b-197c-4d92-89d6-15aec0d48ab5 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:27.550847 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1043/2091] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:38:27 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:38:27.555072 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c777af-98e0-4f93-a45a-586b72f32bd9 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:27.562321 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:38:27.567852 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c777af-98e0-4f93-a45a-586b72f32bd9 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:27.573640 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ee3291-74ac-473a-8eeb-a16364c89abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:27.574272 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ee3291-74ac-473a-8eeb-a16364c89abb admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:27.574625 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ee3291-74ac-473a-8eeb-a16364c89abb admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:27.574990 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ee3291-74ac-473a-8eeb-a16364c89abb admin admin] PATH_INFO: `/v3/projects/d7280ca621b24228b4c729aa57fa88b4/users/918bf5031e8e415ca5ef908a180855a2/roles/28763c6e70094116b3a4b2895895fa58` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:27.592231 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ee3291-74ac-473a-8eeb-a16364c89abb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7280ca621b24228b4c729aa57fa88b4, user_id=918bf5031e8e415ca5ef908a180855a2, role_id=28763c6e70094116b3a4b2895895fa58)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:27.595068 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ee3291-74ac-473a-8eeb-a16364c89abb admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:27.772467 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976e618b-197c-4d92-89d6-15aec0d48ab5 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:27.777343 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-976e618b-197c-4d92-89d6-15aec0d48ab5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:27.785207 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1049/2092] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:38:27 2022] PUT /identity/v3/projects/d7280ca621b24228b4c729aa57fa88b4/users/918bf5031e8e415ca5ef908a180855a2/roles/28763c6e70094116b3a4b2895895fa58 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:38:27.786408 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217fa27c-8f66-41d1-b0ca-a8ef3382deec None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:27.786730 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217fa27c-8f66-41d1-b0ca-a8ef3382deec None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:27.787012 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217fa27c-8f66-41d1-b0ca-a8ef3382deec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:27.811742 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976e618b-197c-4d92-89d6-15aec0d48ab5 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:27.814251 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-217fa27c-8f66-41d1-b0ca-a8ef3382deec None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:38:27.816043 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-217fa27c-8f66-41d1-b0ca-a8ef3382deec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:27.884840 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df07487-f2b8-4a42-b9d1-867685527cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:27.885974 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df07487-f2b8-4a42-b9d1-867685527cb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:27.886254 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df07487-f2b8-4a42-b9d1-867685527cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:27.886813 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df07487-f2b8-4a42-b9d1-867685527cb9 admin admin] PATH_INFO: `/v3/projects/2771cec8973344a493780b7f1cf51e18` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:27.887619 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1050/2093] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:38:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:38:27.888369 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df07487-f2b8-4a42-b9d1-867685527cb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2771cec8973344a493780b7f1cf51e18)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:27.895321 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df07487-f2b8-4a42-b9d1-867685527cb9 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:27.898629 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217fa27c-8f66-41d1-b0ca-a8ef3382deec None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:27.900568 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-217fa27c-8f66-41d1-b0ca-a8ef3382deec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:27.966381 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217fa27c-8f66-41d1-b0ca-a8ef3382deec None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:27.970357 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6ae1ca-aca1-4afb-ba0f-44db9d216099 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:27.975140 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6ae1ca-aca1-4afb-ba0f-44db9d216099 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:27.975140 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6ae1ca-aca1-4afb-ba0f-44db9d216099 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:27.975140 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6ae1ca-aca1-4afb-ba0f-44db9d216099 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:27.981879 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6ae1ca-aca1-4afb-ba0f-44db9d216099 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:27.983452 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6ae1ca-aca1-4afb-ba0f-44db9d216099 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:27.988199 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:38:27.994633 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1051/2094] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:38:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 19:38:28.021485 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3df07487-f2b8-4a42-b9d1-867685527cb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2771cec8973344a493780b7f1cf51e18', 'request_id': 'req-3df07487-f2b8-4a42-b9d1-867685527cb9'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:28.102827 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3df07487-f2b8-4a42-b9d1-867685527cb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2771cec8973344a493780b7f1cf51e18', 'request_id': 'req-3df07487-f2b8-4a42-b9d1-867685527cb9'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:28.109603 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3df07487-f2b8-4a42-b9d1-867685527cb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2771cec8973344a493780b7f1cf51e18', 'request_id': 'req-3df07487-f2b8-4a42-b9d1-867685527cb9'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:28.133380 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3df07487-f2b8-4a42-b9d1-867685527cb9 admin admin] The token cache is being invalidate because project 2771cec8973344a493780b7f1cf51e18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:38:28.133673 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3df07487-f2b8-4a42-b9d1-867685527cb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:28.134723 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3df07487-f2b8-4a42-b9d1-867685527cb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:28.140129 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1044/2095] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jul 15 19:38:27 2022] DELETE /identity/v3/projects/2771cec8973344a493780b7f1cf51e18 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:38:28.142214 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2c09aa-31c6-4644-b1af-eec43ad96d97 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:28.142582 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2c09aa-31c6-4644-b1af-eec43ad96d97 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:28.143051 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2c09aa-31c6-4644-b1af-eec43ad96d97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:28.163869 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-9e2c09aa-31c6-4644-b1af-eec43ad96d97 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:38:28.165361 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-9e2c09aa-31c6-4644-b1af-eec43ad96d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:28.234211 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1052/2096] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:38:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:38:28.242170 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d640a07c-5e0b-4570-ad76-069e2783a554 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:28.242462 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d640a07c-5e0b-4570-ad76-069e2783a554 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:28.242772 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d640a07c-5e0b-4570-ad76-069e2783a554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:28.262858 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-d640a07c-5e0b-4570-ad76-069e2783a554 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:38:28.264220 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d640a07c-5e0b-4570-ad76-069e2783a554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:28.272862 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1045/2097] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:38:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:38:28.277121 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2c09aa-31c6-4644-b1af-eec43ad96d97 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:28.287415 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2c09aa-31c6-4644-b1af-eec43ad96d97 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:28.292099 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4412e18-5fd5-42f5-a14d-cae71e1f97a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:28.292748 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4412e18-5fd5-42f5-a14d-cae71e1f97a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:28.293028 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4412e18-5fd5-42f5-a14d-cae71e1f97a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:28.293327 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4412e18-5fd5-42f5-a14d-cae71e1f97a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:28.294081 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4412e18-5fd5-42f5-a14d-cae71e1f97a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:38:28.294386 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4412e18-5fd5-42f5-a14d-cae71e1f97a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:28.295968 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4412e18-5fd5-42f5-a14d-cae71e1f97a3 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:28.307335 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1053/2098] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jul 15 19:38:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:38:28.311625 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d640a07c-5e0b-4570-ad76-069e2783a554 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:28.321345 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d640a07c-5e0b-4570-ad76-069e2783a554 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:28.326318 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d68682-6546-4b88-bef7-30a4e7c1ca5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:28.327045 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d68682-6546-4b88-bef7-30a4e7c1ca5f admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:28.327341 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d68682-6546-4b88-bef7-30a4e7c1ca5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:28.327635 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d68682-6546-4b88-bef7-30a4e7c1ca5f admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:28.328628 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d68682-6546-4b88-bef7-30a4e7c1ca5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:28.330619 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d68682-6546-4b88-bef7-30a4e7c1ca5f admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:28.482993 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1046/2099] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jul 15 19:38:28 2022] POST /identity/v3/projects => generated 379 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:38:28.484690 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4412e18-5fd5-42f5-a14d-cae71e1f97a3 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:28.495807 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4412e18-5fd5-42f5-a14d-cae71e1f97a3 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:28.559177 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea55ab3-3fc2-46b5-802f-b0a28472d66b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:28.559939 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea55ab3-3fc2-46b5-802f-b0a28472d66b admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:28.560295 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea55ab3-3fc2-46b5-802f-b0a28472d66b admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:28.560636 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea55ab3-3fc2-46b5-802f-b0a28472d66b admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:28.561406 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea55ab3-3fc2-46b5-802f-b0a28472d66b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:28.563459 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea55ab3-3fc2-46b5-802f-b0a28472d66b admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:28.706619 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1054/2100] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jul 15 19:38:28 2022] POST /identity/v3/users => generated 290 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:38:28.716744 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d68682-6546-4b88-bef7-30a4e7c1ca5f admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:28.730409 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d68682-6546-4b88-bef7-30a4e7c1ca5f admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:28.740099 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1cc064-9800-4b81-a436-8976587bd53a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:28.740781 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1cc064-9800-4b81-a436-8976587bd53a admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:28.741066 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1cc064-9800-4b81-a436-8976587bd53a admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:28.741342 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1cc064-9800-4b81-a436-8976587bd53a admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:28.742122 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1cc064-9800-4b81-a436-8976587bd53a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:28.744062 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1cc064-9800-4b81-a436-8976587bd53a admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:28.755702 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1047/2101] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:38:28 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:38:28.760106 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea55ab3-3fc2-46b5-802f-b0a28472d66b admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:28.771873 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea55ab3-3fc2-46b5-802f-b0a28472d66b admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:28.777468 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5405be5b-8a74-47c5-9e87-ccc8de6799c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:28.778136 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5405be5b-8a74-47c5-9e87-ccc8de6799c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:28.782896 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5405be5b-8a74-47c5-9e87-ccc8de6799c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:28.782896 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5405be5b-8a74-47c5-9e87-ccc8de6799c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:28.782896 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5405be5b-8a74-47c5-9e87-ccc8de6799c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:28.782896 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5405be5b-8a74-47c5-9e87-ccc8de6799c2 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:28.794186 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1055/2102] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:38:28 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:38:28.798498 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1cc064-9800-4b81-a436-8976587bd53a admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:28.809421 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1cc064-9800-4b81-a436-8976587bd53a admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:28.815027 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ac12fd-0d82-4c1b-b703-e8306b6a3cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:28.816603 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ac12fd-0d82-4c1b-b703-e8306b6a3cca admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:28.816882 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ac12fd-0d82-4c1b-b703-e8306b6a3cca admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:28.817333 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ac12fd-0d82-4c1b-b703-e8306b6a3cca admin admin] PATH_INFO: `/v3/projects/a832a5ae288d41f1824e57b1bcc6dca1/users/41d65200f05f4096823ecc0229b1ce3b/roles/28763c6e70094116b3a4b2895895fa58` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:28.834893 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ac12fd-0d82-4c1b-b703-e8306b6a3cca admin admin] RBAC: Authorizing `identity:create_grant(project_id=a832a5ae288d41f1824e57b1bcc6dca1, user_id=41d65200f05f4096823ecc0229b1ce3b, role_id=28763c6e70094116b3a4b2895895fa58)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:28.837939 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ac12fd-0d82-4c1b-b703-e8306b6a3cca admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:28.959998 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:38:28.966384 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1048/2103] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:38:28 2022] PUT /identity/v3/projects/a832a5ae288d41f1824e57b1bcc6dca1/users/41d65200f05f4096823ecc0229b1ce3b/roles/28763c6e70094116b3a4b2895895fa58 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:38:28.972552 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e124aff-6f5b-4237-8418-1bff684e70b8 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:28.972841 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e124aff-6f5b-4237-8418-1bff684e70b8 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:28.973116 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e124aff-6f5b-4237-8418-1bff684e70b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:28.990146 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-1e124aff-6f5b-4237-8418-1bff684e70b8 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:38:28.991357 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-1e124aff-6f5b-4237-8418-1bff684e70b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:29.051134 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1056/2104] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:38:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:38:29.060924 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ac12fd-0d82-4c1b-b703-e8306b6a3cca admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:29.062772 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-92ac12fd-0d82-4c1b-b703-e8306b6a3cca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:29.128283 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ac12fd-0d82-4c1b-b703-e8306b6a3cca admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:29.132567 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8524d7e9-aec7-4c03-a341-f315c4e14122 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:29.133620 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8524d7e9-aec7-4c03-a341-f315c4e14122 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:29.133942 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8524d7e9-aec7-4c03-a341-f315c4e14122 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:29.134224 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8524d7e9-aec7-4c03-a341-f315c4e14122 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:29.146080 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8524d7e9-aec7-4c03-a341-f315c4e14122 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:29.147614 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8524d7e9-aec7-4c03-a341-f315c4e14122 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:29.160006 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1049/2105] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:38:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:38:29.571694 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b622fef-6e87-41a5-bd27-beb5dac9e4f8 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:29.571995 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b622fef-6e87-41a5-bd27-beb5dac9e4f8 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:29.572270 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b622fef-6e87-41a5-bd27-beb5dac9e4f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:29.581620 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:38:29.607713 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-3b622fef-6e87-41a5-bd27-beb5dac9e4f8 None None] MFA Rules not processed for user `918bf5031e8e415ca5ef908a180855a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:38:29.609947 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-3b622fef-6e87-41a5-bd27-beb5dac9e4f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:29.685190 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1057/2106] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:38:29 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:38:29.691069 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8524d7e9-aec7-4c03-a341-f315c4e14122 service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:29.692484 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-8524d7e9-aec7-4c03-a341-f315c4e14122 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:29.706507 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8524d7e9-aec7-4c03-a341-f315c4e14122 service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:29.712337 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18f3710-e187-438f-b49b-52f0e36f810a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:29.712337 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18f3710-e187-438f-b49b-52f0e36f810a admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:29.712337 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18f3710-e187-438f-b49b-52f0e36f810a admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:29.712638 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18f3710-e187-438f-b49b-52f0e36f810a admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:29.713205 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18f3710-e187-438f-b49b-52f0e36f810a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:29.715254 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18f3710-e187-438f-b49b-52f0e36f810a admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:29.860929 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1050/2107] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jul 15 19:38:29 2022] POST /identity/v3/projects => generated 373 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:38:29.865706 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b622fef-6e87-41a5-bd27-beb5dac9e4f8 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:29.881599 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b622fef-6e87-41a5-bd27-beb5dac9e4f8 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:29.947689 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae0c6a5-de45-481d-948f-076a606b0c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:29.948375 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae0c6a5-de45-481d-948f-076a606b0c9a admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:29.948657 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae0c6a5-de45-481d-948f-076a606b0c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:29.948933 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae0c6a5-de45-481d-948f-076a606b0c9a admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:29.949627 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae0c6a5-de45-481d-948f-076a606b0c9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:29.951619 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae0c6a5-de45-481d-948f-076a606b0c9a admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:30.095891 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1058/2108] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jul 15 19:38:29 2022] POST /identity/v3/users => generated 293 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:38:30.100083 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18f3710-e187-438f-b49b-52f0e36f810a admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:30.111304 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18f3710-e187-438f-b49b-52f0e36f810a admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:30.116331 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da088ef-d45c-4796-a5a2-329b14ee7a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:30.117094 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da088ef-d45c-4796-a5a2-329b14ee7a7f admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:30.117373 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da088ef-d45c-4796-a5a2-329b14ee7a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:30.117646 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da088ef-d45c-4796-a5a2-329b14ee7a7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:30.118417 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da088ef-d45c-4796-a5a2-329b14ee7a7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:30.120172 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da088ef-d45c-4796-a5a2-329b14ee7a7f admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:30.133199 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1051/2109] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:38:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:38:30.137931 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae0c6a5-de45-481d-948f-076a606b0c9a admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:30.149255 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae0c6a5-de45-481d-948f-076a606b0c9a admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:30.154966 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314675f6-11ea-4cee-9f7b-ace6b8b08e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:30.156504 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314675f6-11ea-4cee-9f7b-ace6b8b08e99 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:30.156785 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314675f6-11ea-4cee-9f7b-ace6b8b08e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:30.157322 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314675f6-11ea-4cee-9f7b-ace6b8b08e99 admin admin] PATH_INFO: `/v3/projects/a44a47a5541743b09e08a5752d42abc1/users/979b97da7c0c40d3a5de6bd165fb6ea1/roles/4318f86ba9f74cd4a0bef1c311276657` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:30.180184 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314675f6-11ea-4cee-9f7b-ace6b8b08e99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a44a47a5541743b09e08a5752d42abc1, user_id=979b97da7c0c40d3a5de6bd165fb6ea1, role_id=4318f86ba9f74cd4a0bef1c311276657)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:30.184025 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314675f6-11ea-4cee-9f7b-ace6b8b08e99 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:30.307177 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1059/2110] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:38:30 2022] PUT /identity/v3/projects/a44a47a5541743b09e08a5752d42abc1/users/979b97da7c0c40d3a5de6bd165fb6ea1/roles/4318f86ba9f74cd4a0bef1c311276657 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:38:30.311642 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da088ef-d45c-4796-a5a2-329b14ee7a7f admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:30.323550 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da088ef-d45c-4796-a5a2-329b14ee7a7f admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:30.382391 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2916f501-0618-4f86-916e-7a0d52880fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:30.383369 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2916f501-0618-4f86-916e-7a0d52880fa5 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:30.383672 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2916f501-0618-4f86-916e-7a0d52880fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:30.383929 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2916f501-0618-4f86-916e-7a0d52880fa5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:30.384743 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2916f501-0618-4f86-916e-7a0d52880fa5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:30.386396 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2916f501-0618-4f86-916e-7a0d52880fa5 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:30.404245 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1052/2111] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:38:30 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:38:30.404834 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314675f6-11ea-4cee-9f7b-ace6b8b08e99 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:30.418900 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314675f6-11ea-4cee-9f7b-ace6b8b08e99 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:30.429086 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632543e8-6a22-4e14-bf5e-da5fb298dbc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:30.430256 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632543e8-6a22-4e14-bf5e-da5fb298dbc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:30.430642 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632543e8-6a22-4e14-bf5e-da5fb298dbc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:30.431045 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632543e8-6a22-4e14-bf5e-da5fb298dbc2 admin admin] PATH_INFO: `/v3/projects/a44a47a5541743b09e08a5752d42abc1/users/979b97da7c0c40d3a5de6bd165fb6ea1/roles/4318f86ba9f74cd4a0bef1c311276657` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:30.433666 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632543e8-6a22-4e14-bf5e-da5fb298dbc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a44a47a5541743b09e08a5752d42abc1, user_id=979b97da7c0c40d3a5de6bd165fb6ea1, role_id=4318f86ba9f74cd4a0bef1c311276657)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:30.437081 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632543e8-6a22-4e14-bf5e-da5fb298dbc2 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:30.453816 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1060/2112] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:38:30 2022] PUT /identity/v3/projects/a44a47a5541743b09e08a5752d42abc1/users/979b97da7c0c40d3a5de6bd165fb6ea1/roles/4318f86ba9f74cd4a0bef1c311276657 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:38:31.634483 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2916f501-0618-4f86-916e-7a0d52880fa5 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:31.636444 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-2916f501-0618-4f86-916e-7a0d52880fa5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:31.650281 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2916f501-0618-4f86-916e-7a0d52880fa5 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:31.758337 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b20a8f-92ef-4f57-9d59-097394ef9c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:31.759522 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b20a8f-92ef-4f57-9d59-097394ef9c30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:31.760290 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b20a8f-92ef-4f57-9d59-097394ef9c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:31.760642 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b20a8f-92ef-4f57-9d59-097394ef9c30 admin admin] PATH_INFO: `/v3/users/79db73eeecb24588b34950fd0b4e828a` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:31.762632 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b20a8f-92ef-4f57-9d59-097394ef9c30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79db73eeecb24588b34950fd0b4e828a)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:31.765352 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b20a8f-92ef-4f57-9d59-097394ef9c30 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:31.793975 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:38:32.141935 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d7b20a8f-92ef-4f57-9d59-097394ef9c30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79db73eeecb24588b34950fd0b4e828a'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:32.154625 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d7b20a8f-92ef-4f57-9d59-097394ef9c30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79db73eeecb24588b34950fd0b4e828a'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:32.295574 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d7b20a8f-92ef-4f57-9d59-097394ef9c30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79db73eeecb24588b34950fd0b4e828a'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:32.296219 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d7b20a8f-92ef-4f57-9d59-097394ef9c30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79db73eeecb24588b34950fd0b4e828a'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:32.304225 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d7b20a8f-92ef-4f57-9d59-097394ef9c30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79db73eeecb24588b34950fd0b4e828a'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:32.310436 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1053/2113] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jul 15 19:38:31 2022] DELETE /identity/v3/users/79db73eeecb24588b34950fd0b4e828a => generated 0 bytes in 677 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:38:32.315984 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd8f92d-9575-45a7-8f03-b0a45790b41c None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:32.316478 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd8f92d-9575-45a7-8f03-b0a45790b41c None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:32.316719 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd8f92d-9575-45a7-8f03-b0a45790b41c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:32.335062 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-4dd8f92d-9575-45a7-8f03-b0a45790b41c None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:38:32.336282 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-4dd8f92d-9575-45a7-8f03-b0a45790b41c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:32.369531 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf07b7dd-aaff-40fc-b41d-58cb5340c33c None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:32.371677 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf07b7dd-aaff-40fc-b41d-58cb5340c33c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:32.372057 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf07b7dd-aaff-40fc-b41d-58cb5340c33c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:32.384169 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:38:32.392068 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-bf07b7dd-aaff-40fc-b41d-58cb5340c33c None None] MFA Rules not processed for user `8a654f77867341a1adf8db7f36ddc653`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:38:32.394746 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-bf07b7dd-aaff-40fc-b41d-58cb5340c33c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:32.411755 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1061/2114] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:38:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:38:32.416395 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd8f92d-9575-45a7-8f03-b0a45790b41c None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:32.418357 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-4dd8f92d-9575-45a7-8f03-b0a45790b41c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:32.436935 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd8f92d-9575-45a7-8f03-b0a45790b41c None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:32.441977 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc92a20a-bf2b-44e2-bec3-da0750213231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:32.443135 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc92a20a-bf2b-44e2-bec3-da0750213231 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:32.443505 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc92a20a-bf2b-44e2-bec3-da0750213231 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:32.443845 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc92a20a-bf2b-44e2-bec3-da0750213231 admin admin] PATH_INFO: `/v3/users/7be011e18a384decac9c016d4251a200` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:32.445176 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc92a20a-bf2b-44e2-bec3-da0750213231 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7be011e18a384decac9c016d4251a200)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:32.447245 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc92a20a-bf2b-44e2-bec3-da0750213231 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:32.482926 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1054/2115] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:38:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:38:32.483607 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf07b7dd-aaff-40fc-b41d-58cb5340c33c None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:32.496771 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-bf07b7dd-aaff-40fc-b41d-58cb5340c33c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:32.569337 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf07b7dd-aaff-40fc-b41d-58cb5340c33c None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:32.576665 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b5ca40-5288-4b58-a302-062b91d1da6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:32.579465 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b5ca40-5288-4b58-a302-062b91d1da6d service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:32.579893 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b5ca40-5288-4b58-a302-062b91d1da6d service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:32.587679 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b5ca40-5288-4b58-a302-062b91d1da6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:32.598436 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b5ca40-5288-4b58-a302-062b91d1da6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:32.599961 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b5ca40-5288-4b58-a302-062b91d1da6d service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:32.611267 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1055/2116] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:38:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:38:32.714066 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec14e02c-709b-420f-b35f-6e82594fa415 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:32.714066 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec14e02c-709b-420f-b35f-6e82594fa415 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:32.714066 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec14e02c-709b-420f-b35f-6e82594fa415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:32.770726 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-ec14e02c-709b-420f-b35f-6e82594fa415 None None] MFA Rules not processed for user `979b97da7c0c40d3a5de6bd165fb6ea1`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:38:32.771895 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-bc92a20a-bf2b-44e2-bec3-da0750213231 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7be011e18a384decac9c016d4251a200'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:32.772022 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-ec14e02c-709b-420f-b35f-6e82594fa415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:32.789579 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-bc92a20a-bf2b-44e2-bec3-da0750213231 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7be011e18a384decac9c016d4251a200'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:32.889964 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1056/2117] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:38:32 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:38:32.901980 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec14e02c-709b-420f-b35f-6e82594fa415 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:32.903165 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-ec14e02c-709b-420f-b35f-6e82594fa415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:32.910160 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-bc92a20a-bf2b-44e2-bec3-da0750213231 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7be011e18a384decac9c016d4251a200'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:32.914622 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-bc92a20a-bf2b-44e2-bec3-da0750213231 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7be011e18a384decac9c016d4251a200'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:32.915938 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-bc92a20a-bf2b-44e2-bec3-da0750213231 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7be011e18a384decac9c016d4251a200'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:32.955196 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1062/2118] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jul 15 19:38:32 2022] DELETE /identity/v3/users/7be011e18a384decac9c016d4251a200 => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:38:32.957348 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de86badf-fdd8-4ae3-a4fe-61c6802c232b None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:32.957724 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de86badf-fdd8-4ae3-a4fe-61c6802c232b None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:32.958101 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de86badf-fdd8-4ae3-a4fe-61c6802c232b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:33.006860 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-de86badf-fdd8-4ae3-a4fe-61c6802c232b None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:38:33.011254 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-de86badf-fdd8-4ae3-a4fe-61c6802c232b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:33.029298 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec14e02c-709b-420f-b35f-6e82594fa415 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:33.095786 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1063/2119] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:38:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:38:33.134113 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d039abe7-b6c2-4feb-89c5-4d315148169e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86c2df3d378490ca7ea497a73c19d40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:33.135566 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d039abe7-b6c2-4feb-89c5-4d315148169e service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:33.135929 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d039abe7-b6c2-4feb-89c5-4d315148169e service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:33.136274 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d039abe7-b6c2-4feb-89c5-4d315148169e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:33.139041 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d039abe7-b6c2-4feb-89c5-4d315148169e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:33.140810 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de86badf-fdd8-4ae3-a4fe-61c6802c232b None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:33.143126 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-de86badf-fdd8-4ae3-a4fe-61c6802c232b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:33.279594 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de86badf-fdd8-4ae3-a4fe-61c6802c232b None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:33.290504 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b1a489-90fb-43a2-b611-9b2512e6879a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:33.291746 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b1a489-90fb-43a2-b611-9b2512e6879a service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:33.292129 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b1a489-90fb-43a2-b611-9b2512e6879a service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:33.293833 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b1a489-90fb-43a2-b611-9b2512e6879a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:33.302613 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d039abe7-b6c2-4feb-89c5-4d315148169e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:33.308834 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d039abe7-b6c2-4feb-89c5-4d315148169e service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:33.317601 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b1a489-90fb-43a2-b611-9b2512e6879a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:33.323176 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b1a489-90fb-43a2-b611-9b2512e6879a service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:33.352538 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1057/2120] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:38:32 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 19:38:33.358618 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1064/2121] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:38:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 19:38:33.456275 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d039abe7-b6c2-4feb-89c5-4d315148169e service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:33.458965 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d039abe7-b6c2-4feb-89c5-4d315148169e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:33.485369 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d039abe7-b6c2-4feb-89c5-4d315148169e service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:33.500878 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8918fe7-ad63-480e-a727-0fe1a820fab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:33.502261 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8918fe7-ad63-480e-a727-0fe1a820fab1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:33.502656 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8918fe7-ad63-480e-a727-0fe1a820fab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:33.503093 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8918fe7-ad63-480e-a727-0fe1a820fab1 admin admin] PATH_INFO: `/v3/projects/ef77f530825441aa8a7cd3113b39670e` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:33.506017 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8918fe7-ad63-480e-a727-0fe1a820fab1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef77f530825441aa8a7cd3113b39670e)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:33.509047 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8918fe7-ad63-480e-a727-0fe1a820fab1 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:33.719487 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e8918fe7-ad63-480e-a727-0fe1a820fab1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef77f530825441aa8a7cd3113b39670e', 'request_id': 'req-e8918fe7-ad63-480e-a727-0fe1a820fab1'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:33.830178 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e8918fe7-ad63-480e-a727-0fe1a820fab1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef77f530825441aa8a7cd3113b39670e', 'request_id': 'req-e8918fe7-ad63-480e-a727-0fe1a820fab1'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:33.837665 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e8918fe7-ad63-480e-a727-0fe1a820fab1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef77f530825441aa8a7cd3113b39670e', 'request_id': 'req-e8918fe7-ad63-480e-a727-0fe1a820fab1'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:33.858354 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e8918fe7-ad63-480e-a727-0fe1a820fab1 admin admin] The token cache is being invalidate because project ef77f530825441aa8a7cd3113b39670e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:38:33.859873 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e8918fe7-ad63-480e-a727-0fe1a820fab1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:33.861535 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e8918fe7-ad63-480e-a727-0fe1a820fab1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:33.866485 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1058/2122] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jul 15 19:38:33 2022] DELETE /identity/v3/projects/ef77f530825441aa8a7cd3113b39670e => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:38:33.871404 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93055093-4890-4697-9b3b-d460176a8b01 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:33.871697 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93055093-4890-4697-9b3b-d460176a8b01 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:33.872005 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93055093-4890-4697-9b3b-d460176a8b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:33.907618 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-93055093-4890-4697-9b3b-d460176a8b01 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:38:33.907618 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-93055093-4890-4697-9b3b-d460176a8b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:34.021738 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1065/2123] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:38:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:38:34.024001 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5ce570-b4c3-4e4e-9c75-5ead408480c1 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:34.024404 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5ce570-b4c3-4e4e-9c75-5ead408480c1 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:34.024961 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5ce570-b4c3-4e4e-9c75-5ead408480c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:34.049083 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93055093-4890-4697-9b3b-d460176a8b01 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:34.050231 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:38:34.051720 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-93055093-4890-4697-9b3b-d460176a8b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:34.069135 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93055093-4890-4697-9b3b-d460176a8b01 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:34.082917 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1f5036-390b-4ce0-9e0e-4c1c42b014cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:34.084181 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1f5036-390b-4ce0-9e0e-4c1c42b014cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:34.084643 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1f5036-390b-4ce0-9e0e-4c1c42b014cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:34.085902 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1f5036-390b-4ce0-9e0e-4c1c42b014cd admin admin] PATH_INFO: `/v3/projects/55a95e8c28764b989f0b454cbac4c30c` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:34.086728 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1f5036-390b-4ce0-9e0e-4c1c42b014cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=55a95e8c28764b989f0b454cbac4c30c)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:34.089231 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1f5036-390b-4ce0-9e0e-4c1c42b014cd admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:34.110882 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-6c5ce570-b4c3-4e4e-9c75-5ead408480c1 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:38:34.116837 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-6c5ce570-b4c3-4e4e-9c75-5ead408480c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:34.149986 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1059/2124] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:38:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:38:34.154491 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5ce570-b4c3-4e4e-9c75-5ead408480c1 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:34.166168 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5ce570-b4c3-4e4e-9c75-5ead408480c1 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:34.171340 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3223bc9-3b1d-41d1-8e44-c348e251b3b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:34.172024 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3223bc9-3b1d-41d1-8e44-c348e251b3b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:34.172307 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3223bc9-3b1d-41d1-8e44-c348e251b3b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:34.172767 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3223bc9-3b1d-41d1-8e44-c348e251b3b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:34.173518 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3223bc9-3b1d-41d1-8e44-c348e251b3b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:38:34.173855 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3223bc9-3b1d-41d1-8e44-c348e251b3b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:34.176801 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3223bc9-3b1d-41d1-8e44-c348e251b3b0 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:34.189845 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1060/2125] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jul 15 19:38:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:38:34.195104 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3223bc9-3b1d-41d1-8e44-c348e251b3b0 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:34.208401 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3223bc9-3b1d-41d1-8e44-c348e251b3b0 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:34.214009 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f1ec32-84c8-4d9c-a9cd-bced3260407a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:34.214633 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f1ec32-84c8-4d9c-a9cd-bced3260407a admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:34.214936 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f1ec32-84c8-4d9c-a9cd-bced3260407a admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:34.215193 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f1ec32-84c8-4d9c-a9cd-bced3260407a admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:34.215894 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f1ec32-84c8-4d9c-a9cd-bced3260407a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:34.218133 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f1ec32-84c8-4d9c-a9cd-bced3260407a admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:34.279541 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-8a1f5036-390b-4ce0-9e0e-4c1c42b014cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55a95e8c28764b989f0b454cbac4c30c', 'request_id': 'req-8a1f5036-390b-4ce0-9e0e-4c1c42b014cd'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:34.419558 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-8a1f5036-390b-4ce0-9e0e-4c1c42b014cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55a95e8c28764b989f0b454cbac4c30c', 'request_id': 'req-8a1f5036-390b-4ce0-9e0e-4c1c42b014cd'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:34.437941 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1061/2126] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jul 15 19:38:34 2022] POST /identity/v3/projects => generated 375 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:38:34.438763 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-8a1f5036-390b-4ce0-9e0e-4c1c42b014cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55a95e8c28764b989f0b454cbac4c30c', 'request_id': 'req-8a1f5036-390b-4ce0-9e0e-4c1c42b014cd'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:34.438884 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f1ec32-84c8-4d9c-a9cd-bced3260407a admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:34.464186 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-8a1f5036-390b-4ce0-9e0e-4c1c42b014cd admin admin] The token cache is being invalidate because project 55a95e8c28764b989f0b454cbac4c30c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:38:34.464526 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-8a1f5036-390b-4ce0-9e0e-4c1c42b014cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:34.467273 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-8a1f5036-390b-4ce0-9e0e-4c1c42b014cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:34.470292 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1066/2127] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jul 15 19:38:34 2022] DELETE /identity/v3/projects/55a95e8c28764b989f0b454cbac4c30c => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:38:34.478545 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bc9e3d-f886-4cd3-a227-26b0ec51dee9 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:34.478788 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bc9e3d-f886-4cd3-a227-26b0ec51dee9 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:34.479012 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bc9e3d-f886-4cd3-a227-26b0ec51dee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:34.519697 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-84bc9e3d-f886-4cd3-a227-26b0ec51dee9 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:38:34.521336 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-84bc9e3d-f886-4cd3-a227-26b0ec51dee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:34.622679 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f1ec32-84c8-4d9c-a9cd-bced3260407a admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:34.635165 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a01a1d-63c4-4af3-b51e-1400382ab070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:34.636387 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a01a1d-63c4-4af3-b51e-1400382ab070 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:34.636830 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a01a1d-63c4-4af3-b51e-1400382ab070 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:34.637256 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a01a1d-63c4-4af3-b51e-1400382ab070 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:34.639412 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a01a1d-63c4-4af3-b51e-1400382ab070 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:34.644192 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a01a1d-63c4-4af3-b51e-1400382ab070 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:34.665749 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1067/2128] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:38:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:38:34.672693 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ca6fd4-f164-41e8-8cce-6d7fba3fbb03 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:34.673164 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ca6fd4-f164-41e8-8cce-6d7fba3fbb03 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:34.673591 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ca6fd4-f164-41e8-8cce-6d7fba3fbb03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:34.737409 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-63ca6fd4-f164-41e8-8cce-6d7fba3fbb03 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:38:34.743167 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-63ca6fd4-f164-41e8-8cce-6d7fba3fbb03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:34.754993 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1068/2129] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:38:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:38:34.759962 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ca6fd4-f164-41e8-8cce-6d7fba3fbb03 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:34.780174 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ca6fd4-f164-41e8-8cce-6d7fba3fbb03 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:34.792708 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdaa24d-3aa8-40ba-9794-830cb09bbb10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:34.793649 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdaa24d-3aa8-40ba-9794-830cb09bbb10 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:34.794181 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdaa24d-3aa8-40ba-9794-830cb09bbb10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:34.794556 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdaa24d-3aa8-40ba-9794-830cb09bbb10 admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:34.795738 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdaa24d-3aa8-40ba-9794-830cb09bbb10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:38:34.796970 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdaa24d-3aa8-40ba-9794-830cb09bbb10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:34.798720 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdaa24d-3aa8-40ba-9794-830cb09bbb10 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:34.809591 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1062/2130] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jul 15 19:38:34 2022] POST /identity/v3/users => generated 288 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:38:34.810917 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a01a1d-63c4-4af3-b51e-1400382ab070 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:34.813236 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-27a01a1d-63c4-4af3-b51e-1400382ab070 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:34.834996 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:38:34.837336 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1069/2131] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jul 15 19:38:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:38:34.838546 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a01a1d-63c4-4af3-b51e-1400382ab070 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:34.840465 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdaa24d-3aa8-40ba-9794-830cb09bbb10 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:34.845199 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8a8fc8-b90d-47d8-8ae7-1843f003a672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:34.847000 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8a8fc8-b90d-47d8-8ae7-1843f003a672 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:34.847529 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8a8fc8-b90d-47d8-8ae7-1843f003a672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:34.848393 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8a8fc8-b90d-47d8-8ae7-1843f003a672 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:34.849552 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8a8fc8-b90d-47d8-8ae7-1843f003a672 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:34.852506 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8a8fc8-b90d-47d8-8ae7-1843f003a672 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:34.855558 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdaa24d-3aa8-40ba-9794-830cb09bbb10 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:34.860804 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede23daf-a341-4c9f-a8f0-f9571f0e6fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:34.861568 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede23daf-a341-4c9f-a8f0-f9571f0e6fbc admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:34.862064 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede23daf-a341-4c9f-a8f0-f9571f0e6fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:34.862539 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede23daf-a341-4c9f-a8f0-f9571f0e6fbc admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:34.863459 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede23daf-a341-4c9f-a8f0-f9571f0e6fbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:34.865624 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede23daf-a341-4c9f-a8f0-f9571f0e6fbc admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:34.867427 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1063/2132] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:38:34 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:38:34.872185 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8a8fc8-b90d-47d8-8ae7-1843f003a672 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:34.885994 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8a8fc8-b90d-47d8-8ae7-1843f003a672 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:34.892632 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55802c47-23c6-4f99-bbf8-a85f6121b69b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:34.894161 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55802c47-23c6-4f99-bbf8-a85f6121b69b admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:34.894673 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55802c47-23c6-4f99-bbf8-a85f6121b69b admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:34.895033 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55802c47-23c6-4f99-bbf8-a85f6121b69b admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:34.895824 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55802c47-23c6-4f99-bbf8-a85f6121b69b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:34.897516 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55802c47-23c6-4f99-bbf8-a85f6121b69b admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:34.910204 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1064/2133] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:38:34 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:38:34.914159 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55802c47-23c6-4f99-bbf8-a85f6121b69b admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:34.925391 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55802c47-23c6-4f99-bbf8-a85f6121b69b admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:34.930274 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c046ff-4249-48a8-8e25-5c8cfc203cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:34.931161 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c046ff-4249-48a8-8e25-5c8cfc203cc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:34.931448 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c046ff-4249-48a8-8e25-5c8cfc203cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:34.931728 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c046ff-4249-48a8-8e25-5c8cfc203cc9 admin admin] PATH_INFO: `/v3/projects/e2e1abaaf77c443096c57263c1451cb2/users/7337f992172c4a4bb1cb778d9cb09284/roles/28763c6e70094116b3a4b2895895fa58` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:34.950336 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c046ff-4249-48a8-8e25-5c8cfc203cc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2e1abaaf77c443096c57263c1451cb2, user_id=7337f992172c4a4bb1cb778d9cb09284, role_id=28763c6e70094116b3a4b2895895fa58)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:34.953467 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c046ff-4249-48a8-8e25-5c8cfc203cc9 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:35.079449 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1070/2134] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jul 15 19:38:34 2022] POST /identity/v3/projects => generated 375 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:38:35.083482 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede23daf-a341-4c9f-a8f0-f9571f0e6fbc admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:35.124299 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1065/2135] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:38:34 2022] PUT /identity/v3/projects/e2e1abaaf77c443096c57263c1451cb2/users/7337f992172c4a4bb1cb778d9cb09284/roles/28763c6e70094116b3a4b2895895fa58 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:38:35.134913 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab56309-3952-473c-95ea-4acf5cf4717a None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:35.135572 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab56309-3952-473c-95ea-4acf5cf4717a None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:35.135740 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab56309-3952-473c-95ea-4acf5cf4717a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:35.191918 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-4ab56309-3952-473c-95ea-4acf5cf4717a None None] MFA Rules not processed for user `7337f992172c4a4bb1cb778d9cb09284`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:38:35.193438 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-4ab56309-3952-473c-95ea-4acf5cf4717a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:35.215473 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede23daf-a341-4c9f-a8f0-f9571f0e6fbc admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:35.285004 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faf8b13-3672-4747-9894-b44b82288054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:35.285874 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faf8b13-3672-4747-9894-b44b82288054 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:35.286609 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faf8b13-3672-4747-9894-b44b82288054 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:35.286609 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faf8b13-3672-4747-9894-b44b82288054 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:35.287319 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faf8b13-3672-4747-9894-b44b82288054 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:35.289376 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faf8b13-3672-4747-9894-b44b82288054 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:35.298469 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1066/2136] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:38:35 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:38:35.303264 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab56309-3952-473c-95ea-4acf5cf4717a None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:35.314259 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab56309-3952-473c-95ea-4acf5cf4717a None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:35.319662 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fab4e9-0b49-47b8-9510-7de9946bf4ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:35.320323 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fab4e9-0b49-47b8-9510-7de9946bf4ba admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:35.326960 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fab4e9-0b49-47b8-9510-7de9946bf4ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:35.326960 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fab4e9-0b49-47b8-9510-7de9946bf4ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:35.326960 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fab4e9-0b49-47b8-9510-7de9946bf4ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:35.326960 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fab4e9-0b49-47b8-9510-7de9946bf4ba admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:35.446781 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1071/2137] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jul 15 19:38:35 2022] POST /identity/v3/users => generated 288 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:38:35.447893 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faf8b13-3672-4747-9894-b44b82288054 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:35.458541 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faf8b13-3672-4747-9894-b44b82288054 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:35.463982 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6dc3b8-2e06-4926-9c17-56839b1b1fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:35.464277 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6dc3b8-2e06-4926-9c17-56839b1b1fd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:35.464550 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6dc3b8-2e06-4926-9c17-56839b1b1fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:35.464830 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6dc3b8-2e06-4926-9c17-56839b1b1fd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:35.465568 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6dc3b8-2e06-4926-9c17-56839b1b1fd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:35.467432 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6dc3b8-2e06-4926-9c17-56839b1b1fd2 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:35.470593 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1067/2138] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jul 15 19:38:35 2022] POST /identity/v3/projects => generated 371 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:38:35.481552 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fab4e9-0b49-47b8-9510-7de9946bf4ba admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:35.482131 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1072/2139] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:38:35 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:38:35.483093 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6dc3b8-2e06-4926-9c17-56839b1b1fd2 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:35.496074 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6dc3b8-2e06-4926-9c17-56839b1b1fd2 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:35.604857 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626a270c-4b94-418d-8d5f-15b51eac2e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:35.606826 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626a270c-4b94-418d-8d5f-15b51eac2e6c admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:35.607421 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626a270c-4b94-418d-8d5f-15b51eac2e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:35.607981 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626a270c-4b94-418d-8d5f-15b51eac2e6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:35.609746 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626a270c-4b94-418d-8d5f-15b51eac2e6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:35.612536 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626a270c-4b94-418d-8d5f-15b51eac2e6c admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:35.628058 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1073/2140] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:38:35 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:38:35.634207 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626a270c-4b94-418d-8d5f-15b51eac2e6c admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:35.649167 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626a270c-4b94-418d-8d5f-15b51eac2e6c admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:35.649664 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fab4e9-0b49-47b8-9510-7de9946bf4ba admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:35.655651 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312f394e-4be9-4ced-8169-4efd4f6eb52e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:35.657296 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f394e-4be9-4ced-8169-4efd4f6eb52e admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:35.657689 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f394e-4be9-4ced-8169-4efd4f6eb52e admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:35.658079 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f394e-4be9-4ced-8169-4efd4f6eb52e admin admin] PATH_INFO: `/v3/projects/2a9551803e964e3697091197d476568a/users/ee06c7a2e27845388f66444515f03845/roles/28763c6e70094116b3a4b2895895fa58` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:35.661212 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec81248-dfa1-4335-87c8-c05f13a5d944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:35.662309 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec81248-dfa1-4335-87c8-c05f13a5d944 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:35.663709 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec81248-dfa1-4335-87c8-c05f13a5d944 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:35.665025 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec81248-dfa1-4335-87c8-c05f13a5d944 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:35.665972 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec81248-dfa1-4335-87c8-c05f13a5d944 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:35.668190 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec81248-dfa1-4335-87c8-c05f13a5d944 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:35.688977 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312f394e-4be9-4ced-8169-4efd4f6eb52e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a9551803e964e3697091197d476568a, user_id=ee06c7a2e27845388f66444515f03845, role_id=28763c6e70094116b3a4b2895895fa58)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:35.691878 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312f394e-4be9-4ced-8169-4efd4f6eb52e admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:35.833444 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1074/2142] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:38:35 2022] PUT /identity/v3/projects/2a9551803e964e3697091197d476568a/users/ee06c7a2e27845388f66444515f03845/roles/28763c6e70094116b3a4b2895895fa58 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:38:35.837086 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312f394e-4be9-4ced-8169-4efd4f6eb52e admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:35.844411 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1068/2142] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jul 15 19:38:35 2022] POST /identity/v3/users => generated 292 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:38:35.846767 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd70ce79-d4f0-4640-8a41-f4f8b61df397 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:35.847273 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd70ce79-d4f0-4640-8a41-f4f8b61df397 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:35.847712 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd70ce79-d4f0-4640-8a41-f4f8b61df397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:35.895841 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-cd70ce79-d4f0-4640-8a41-f4f8b61df397 None None] MFA Rules not processed for user `ee06c7a2e27845388f66444515f03845`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:38:35.897364 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-cd70ce79-d4f0-4640-8a41-f4f8b61df397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:35.948537 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312f394e-4be9-4ced-8169-4efd4f6eb52e admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:35.954680 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fb77c8-36cf-4902-adbb-65dffb1031fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:35.964918 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fb77c8-36cf-4902-adbb-65dffb1031fb admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:35.965394 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fb77c8-36cf-4902-adbb-65dffb1031fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:35.965979 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fb77c8-36cf-4902-adbb-65dffb1031fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:35.966916 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fb77c8-36cf-4902-adbb-65dffb1031fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:35.968887 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fb77c8-36cf-4902-adbb-65dffb1031fb admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:35.989431 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1075/2143] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:38:35 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 19:38:35.994745 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fb77c8-36cf-4902-adbb-65dffb1031fb admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:36.027209 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fb77c8-36cf-4902-adbb-65dffb1031fb admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:36.027407 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1069/2144] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:38:35 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:38:36.027512 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c0bc5b-5710-490c-a401-d8f64ce90ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:36.027512 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c0bc5b-5710-490c-a401-d8f64ce90ff8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:36.027512 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c0bc5b-5710-490c-a401-d8f64ce90ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:36.027512 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c0bc5b-5710-490c-a401-d8f64ce90ff8 admin admin] PATH_INFO: `/v3/projects/f311050b5ed44425a4856a5c2aa3360d/users/cdfa7019c4054a48a4f0fdb4bc6ef2e6/roles/4318f86ba9f74cd4a0bef1c311276657` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:36.041100 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd70ce79-d4f0-4640-8a41-f4f8b61df397 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:36.042598 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-cd70ce79-d4f0-4640-8a41-f4f8b61df397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:36.059187 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c0bc5b-5710-490c-a401-d8f64ce90ff8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f311050b5ed44425a4856a5c2aa3360d, user_id=cdfa7019c4054a48a4f0fdb4bc6ef2e6, role_id=4318f86ba9f74cd4a0bef1c311276657)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:36.063397 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c0bc5b-5710-490c-a401-d8f64ce90ff8 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:36.167173 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd70ce79-d4f0-4640-8a41-f4f8b61df397 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:36.174158 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31bb34c-cec1-422f-b2e5-898d5370a1da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7673a3b5c0324e07a9535ac70ed7a491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:36.174848 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31bb34c-cec1-422f-b2e5-898d5370a1da service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:36.175641 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31bb34c-cec1-422f-b2e5-898d5370a1da service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:36.175641 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31bb34c-cec1-422f-b2e5-898d5370a1da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:36.184319 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31bb34c-cec1-422f-b2e5-898d5370a1da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:36.204749 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31bb34c-cec1-422f-b2e5-898d5370a1da service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:36.204749 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1070/2145] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:38:36 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 19:38:36.288959 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1076/2146] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:38:35 2022] PUT /identity/v3/projects/f311050b5ed44425a4856a5c2aa3360d/users/cdfa7019c4054a48a4f0fdb4bc6ef2e6/roles/4318f86ba9f74cd4a0bef1c311276657 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:38:36.297314 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31bb34c-cec1-422f-b2e5-898d5370a1da service nova] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:36.314128 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31bb34c-cec1-422f-b2e5-898d5370a1da service nova] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:36.408674 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cd249a-6eff-4405-8978-3796ffa172ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:36.409435 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cd249a-6eff-4405-8978-3796ffa172ec admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:36.409809 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cd249a-6eff-4405-8978-3796ffa172ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:36.410186 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cd249a-6eff-4405-8978-3796ffa172ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:36.410937 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cd249a-6eff-4405-8978-3796ffa172ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:36.412551 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cd249a-6eff-4405-8978-3796ffa172ec admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:36.423398 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1071/2147] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:38:36 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:38:36.428154 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c0bc5b-5710-490c-a401-d8f64ce90ff8 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:36.446613 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c0bc5b-5710-490c-a401-d8f64ce90ff8 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:36.453718 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619b6644-203a-4dc6-b0e2-cd077f40e1f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:36.460303 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b6644-203a-4dc6-b0e2-cd077f40e1f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:36.460303 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b6644-203a-4dc6-b0e2-cd077f40e1f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:36.460303 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b6644-203a-4dc6-b0e2-cd077f40e1f4 admin admin] PATH_INFO: `/v3/projects/f311050b5ed44425a4856a5c2aa3360d/users/cdfa7019c4054a48a4f0fdb4bc6ef2e6/roles/4318f86ba9f74cd4a0bef1c311276657` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:36.460303 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619b6644-203a-4dc6-b0e2-cd077f40e1f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f311050b5ed44425a4856a5c2aa3360d, user_id=cdfa7019c4054a48a4f0fdb4bc6ef2e6, role_id=4318f86ba9f74cd4a0bef1c311276657)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:36.461247 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619b6644-203a-4dc6-b0e2-cd077f40e1f4 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:36.497087 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1077/2148] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:38:36 2022] PUT /identity/v3/projects/f311050b5ed44425a4856a5c2aa3360d/users/cdfa7019c4054a48a4f0fdb4bc6ef2e6/roles/4318f86ba9f74cd4a0bef1c311276657 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:38:36.506224 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b2c826-07e7-4401-8fbc-fd9debf20b5e None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:36.506224 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b2c826-07e7-4401-8fbc-fd9debf20b5e None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:36.506458 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b2c826-07e7-4401-8fbc-fd9debf20b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:36.514093 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619b6644-203a-4dc6-b0e2-cd077f40e1f4 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:36.541874 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-51b2c826-07e7-4401-8fbc-fd9debf20b5e None None] MFA Rules not processed for user `cdfa7019c4054a48a4f0fdb4bc6ef2e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:38:36.543109 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-51b2c826-07e7-4401-8fbc-fd9debf20b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:36.596638 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619b6644-203a-4dc6-b0e2-cd077f40e1f4 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:36.603344 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb59515-7b1d-476f-853a-da3595d6eccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:36.604233 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb59515-7b1d-476f-853a-da3595d6eccf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:36.604567 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb59515-7b1d-476f-853a-da3595d6eccf admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:36.604833 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb59515-7b1d-476f-853a-da3595d6eccf admin admin] PATH_INFO: `/v3/users/ee06c7a2e27845388f66444515f03845` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:36.606491 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb59515-7b1d-476f-853a-da3595d6eccf admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee06c7a2e27845388f66444515f03845)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:36.608544 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb59515-7b1d-476f-853a-da3595d6eccf admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:36.705468 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1072/2149] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:38:36 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:38:36.732676 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b2c826-07e7-4401-8fbc-fd9debf20b5e None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:36.741287 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-51b2c826-07e7-4401-8fbc-fd9debf20b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:36.838417 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b2c826-07e7-4401-8fbc-fd9debf20b5e None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:36.851229 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d027744-445c-40ae-80c7-a88381e56b81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86c2df3d378490ca7ea497a73c19d40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:36.852068 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d027744-445c-40ae-80c7-a88381e56b81 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:36.852793 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d027744-445c-40ae-80c7-a88381e56b81 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:36.853290 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d027744-445c-40ae-80c7-a88381e56b81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:36.896877 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-6eb59515-7b1d-476f-853a-da3595d6eccf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee06c7a2e27845388f66444515f03845'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:36.921544 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-6eb59515-7b1d-476f-853a-da3595d6eccf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee06c7a2e27845388f66444515f03845'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:36.942390 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d027744-445c-40ae-80c7-a88381e56b81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:36.944204 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d027744-445c-40ae-80c7-a88381e56b81 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:36.954548 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1073/2150] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:38:36 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:38:36.956321 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d027744-445c-40ae-80c7-a88381e56b81 service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:36.958019 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-1d027744-445c-40ae-80c7-a88381e56b81 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:36.971647 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d027744-445c-40ae-80c7-a88381e56b81 service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:36.976349 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d409d2-2eb1-4e8d-a3d3-464f62d27467 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86c2df3d378490ca7ea497a73c19d40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:36.977156 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d409d2-2eb1-4e8d-a3d3-464f62d27467 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:36.977445 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d409d2-2eb1-4e8d-a3d3-464f62d27467 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:36.977724 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d409d2-2eb1-4e8d-a3d3-464f62d27467 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:36.983939 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-c8d409d2-2eb1-4e8d-a3d3-464f62d27467 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:37.001662 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-6eb59515-7b1d-476f-853a-da3595d6eccf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee06c7a2e27845388f66444515f03845'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:37.002620 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-6eb59515-7b1d-476f-853a-da3595d6eccf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee06c7a2e27845388f66444515f03845'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:37.003792 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-6eb59515-7b1d-476f-853a-da3595d6eccf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee06c7a2e27845388f66444515f03845'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:37.024421 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1078/2151] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jul 15 19:38:36 2022] DELETE /identity/v3/users/ee06c7a2e27845388f66444515f03845 => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:38:37.044753 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f2f06b-f1b3-4aaf-b934-c15a4d470d4d None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:37.044753 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f2f06b-f1b3-4aaf-b934-c15a4d470d4d None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:37.044753 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f2f06b-f1b3-4aaf-b934-c15a4d470d4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:37.076774 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-67f2f06b-f1b3-4aaf-b934-c15a4d470d4d None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:38:37.080208 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-67f2f06b-f1b3-4aaf-b934-c15a4d470d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:37.105268 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d409d2-2eb1-4e8d-a3d3-464f62d27467 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:37.107028 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d409d2-2eb1-4e8d-a3d3-464f62d27467 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:37.119814 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1074/2152] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:38:36 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:38:37.121369 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72011ec-644c-4aad-9982-5152bceefdfe None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:37.122054 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72011ec-644c-4aad-9982-5152bceefdfe None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:37.122484 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72011ec-644c-4aad-9982-5152bceefdfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:37.185124 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-c72011ec-644c-4aad-9982-5152bceefdfe None None] MFA Rules not processed for user `41d65200f05f4096823ecc0229b1ce3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:38:37.187391 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-c72011ec-644c-4aad-9982-5152bceefdfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:37.228329 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1079/2153] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:38:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:38:37.238378 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f2f06b-f1b3-4aaf-b934-c15a4d470d4d None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:37.240048 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-67f2f06b-f1b3-4aaf-b934-c15a4d470d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:37.293844 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1075/2154] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:38:37 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:38:37.294883 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72011ec-644c-4aad-9982-5152bceefdfe None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:37.296386 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-c72011ec-644c-4aad-9982-5152bceefdfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:37.318908 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72011ec-644c-4aad-9982-5152bceefdfe None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:37.329479 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb8e8ea-243f-4b43-91a8-3725280628be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:37.341741 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb8e8ea-243f-4b43-91a8-3725280628be admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:37.344060 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb8e8ea-243f-4b43-91a8-3725280628be admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:37.344060 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb8e8ea-243f-4b43-91a8-3725280628be admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:37.344060 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb8e8ea-243f-4b43-91a8-3725280628be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:37.346793 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb8e8ea-243f-4b43-91a8-3725280628be admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:37.382189 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f2f06b-f1b3-4aaf-b934-c15a4d470d4d None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:37.388791 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d31eb4-2e23-4d75-8ef0-6a04b227e006 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:37.392457 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d31eb4-2e23-4d75-8ef0-6a04b227e006 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:37.393300 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d31eb4-2e23-4d75-8ef0-6a04b227e006 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:37.393740 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d31eb4-2e23-4d75-8ef0-6a04b227e006 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:37.409367 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d31eb4-2e23-4d75-8ef0-6a04b227e006 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:37.412732 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d31eb4-2e23-4d75-8ef0-6a04b227e006 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:37.429101 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1080/2155] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:38:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 19:38:37.506844 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1076/2156] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jul 15 19:38:37 2022] POST /identity/v3/projects => generated 377 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:38:37.512529 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d31eb4-2e23-4d75-8ef0-6a04b227e006 service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:37.527700 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d31eb4-2e23-4d75-8ef0-6a04b227e006 service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:37.591711 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4204ac8-aef6-42f0-a111-514780aa740d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:37.592518 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4204ac8-aef6-42f0-a111-514780aa740d admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:37.592874 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4204ac8-aef6-42f0-a111-514780aa740d admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:37.593250 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4204ac8-aef6-42f0-a111-514780aa740d admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:37.594086 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4204ac8-aef6-42f0-a111-514780aa740d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:37.596898 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4204ac8-aef6-42f0-a111-514780aa740d admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:37.811618 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb8e8ea-243f-4b43-91a8-3725280628be admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:37.837255 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1081/2157] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jul 15 19:38:37 2022] POST /identity/v3/users => generated 295 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:38:37.837365 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb8e8ea-243f-4b43-91a8-3725280628be admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:37.847210 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36130cb-a2c1-4f36-a59a-8d8ccef8057f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:37.849531 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36130cb-a2c1-4f36-a59a-8d8ccef8057f admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:37.850040 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36130cb-a2c1-4f36-a59a-8d8ccef8057f admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:37.850514 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36130cb-a2c1-4f36-a59a-8d8ccef8057f admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:37.851608 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36130cb-a2c1-4f36-a59a-8d8ccef8057f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:37.853740 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36130cb-a2c1-4f36-a59a-8d8ccef8057f admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:37.883160 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1077/2158] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:38:37 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:38:37.885992 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4204ac8-aef6-42f0-a111-514780aa740d admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:37.897968 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4204ac8-aef6-42f0-a111-514780aa740d admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:37.903248 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3036d17d-8d29-4dd6-88fb-b3ca1e222ff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:37.904122 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3036d17d-8d29-4dd6-88fb-b3ca1e222ff5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:37.904122 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3036d17d-8d29-4dd6-88fb-b3ca1e222ff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:37.904122 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3036d17d-8d29-4dd6-88fb-b3ca1e222ff5 admin admin] PATH_INFO: `/v3/projects/9c36706f378f445ea2d3e6b93e08e6ca/users/f8fed2fa41894eaeafa009161d8bc537/roles/4318f86ba9f74cd4a0bef1c311276657` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:37.922539 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3036d17d-8d29-4dd6-88fb-b3ca1e222ff5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c36706f378f445ea2d3e6b93e08e6ca, user_id=f8fed2fa41894eaeafa009161d8bc537, role_id=4318f86ba9f74cd4a0bef1c311276657)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:37.925392 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3036d17d-8d29-4dd6-88fb-b3ca1e222ff5 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:38.045852 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1082/2159] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:38:37 2022] PUT /identity/v3/projects/9c36706f378f445ea2d3e6b93e08e6ca/users/f8fed2fa41894eaeafa009161d8bc537/roles/4318f86ba9f74cd4a0bef1c311276657 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:38:38.047194 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36130cb-a2c1-4f36-a59a-8d8ccef8057f admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:38.067253 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36130cb-a2c1-4f36-a59a-8d8ccef8057f admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:38.122551 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaceaf44-7e77-4119-b9e3-14fa979a49b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:38.123075 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3036d17d-8d29-4dd6-88fb-b3ca1e222ff5 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:38.123448 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaceaf44-7e77-4119-b9e3-14fa979a49b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:38.124484 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaceaf44-7e77-4119-b9e3-14fa979a49b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:38.124766 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaceaf44-7e77-4119-b9e3-14fa979a49b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:38.124929 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-3036d17d-8d29-4dd6-88fb-b3ca1e222ff5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:38.125512 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaceaf44-7e77-4119-b9e3-14fa979a49b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:38.128304 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaceaf44-7e77-4119-b9e3-14fa979a49b0 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:38.151574 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3036d17d-8d29-4dd6-88fb-b3ca1e222ff5 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:38.161846 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1078/2160] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:38:38 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:38:38.165745 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaceaf44-7e77-4119-b9e3-14fa979a49b0 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:38.168831 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e037db-50cb-4a18-9b6a-aa00c8805de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:38.169740 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e037db-50cb-4a18-9b6a-aa00c8805de7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:38.170057 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e037db-50cb-4a18-9b6a-aa00c8805de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:38.170335 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e037db-50cb-4a18-9b6a-aa00c8805de7 admin admin] PATH_INFO: `/v3/projects/2a9551803e964e3697091197d476568a` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:38.171825 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e037db-50cb-4a18-9b6a-aa00c8805de7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a9551803e964e3697091197d476568a)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:38.173953 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e037db-50cb-4a18-9b6a-aa00c8805de7 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:38.189024 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaceaf44-7e77-4119-b9e3-14fa979a49b0 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:38.194746 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36d8a49-e14b-43d8-b218-39b5f337fd31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:38.195915 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36d8a49-e14b-43d8-b218-39b5f337fd31 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:38.196277 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36d8a49-e14b-43d8-b218-39b5f337fd31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:38.196701 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36d8a49-e14b-43d8-b218-39b5f337fd31 admin admin] PATH_INFO: `/v3/projects/9c36706f378f445ea2d3e6b93e08e6ca/users/f8fed2fa41894eaeafa009161d8bc537/roles/4318f86ba9f74cd4a0bef1c311276657` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:38.199503 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36d8a49-e14b-43d8-b218-39b5f337fd31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c36706f378f445ea2d3e6b93e08e6ca, user_id=f8fed2fa41894eaeafa009161d8bc537, role_id=4318f86ba9f74cd4a0bef1c311276657)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:38.202509 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36d8a49-e14b-43d8-b218-39b5f337fd31 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:38.223331 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1079/2161] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:38:38 2022] PUT /identity/v3/projects/9c36706f378f445ea2d3e6b93e08e6ca/users/f8fed2fa41894eaeafa009161d8bc537/roles/4318f86ba9f74cd4a0bef1c311276657 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:38:38.313091 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-20e037db-50cb-4a18-9b6a-aa00c8805de7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a9551803e964e3697091197d476568a', 'request_id': 'req-20e037db-50cb-4a18-9b6a-aa00c8805de7'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:38.407663 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-20e037db-50cb-4a18-9b6a-aa00c8805de7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a9551803e964e3697091197d476568a', 'request_id': 'req-20e037db-50cb-4a18-9b6a-aa00c8805de7'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:38.416128 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-20e037db-50cb-4a18-9b6a-aa00c8805de7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a9551803e964e3697091197d476568a', 'request_id': 'req-20e037db-50cb-4a18-9b6a-aa00c8805de7'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:38.429467 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-20e037db-50cb-4a18-9b6a-aa00c8805de7 admin admin] The token cache is being invalidate because project 2a9551803e964e3697091197d476568a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:38:38.429749 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-20e037db-50cb-4a18-9b6a-aa00c8805de7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:38.430461 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-20e037db-50cb-4a18-9b6a-aa00c8805de7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:38.432342 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1083/2162] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jul 15 19:38:38 2022] DELETE /identity/v3/projects/2a9551803e964e3697091197d476568a => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:38:38.437355 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e914a51-1e15-4155-a4b8-013286079b66 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:38.437653 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e914a51-1e15-4155-a4b8-013286079b66 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:38.437978 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e914a51-1e15-4155-a4b8-013286079b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:38.461538 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-3e914a51-1e15-4155-a4b8-013286079b66 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:38:38.462806 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-3e914a51-1e15-4155-a4b8-013286079b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:38.562878 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:38:38.597170 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1080/2163] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:38:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:38:38.604531 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba67b4b-042d-4bde-ac6b-3541c42fb90f None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:38.606001 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba67b4b-042d-4bde-ac6b-3541c42fb90f None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:38.606657 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba67b4b-042d-4bde-ac6b-3541c42fb90f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:38.640450 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-bba67b4b-042d-4bde-ac6b-3541c42fb90f None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:38:38.642539 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-bba67b4b-042d-4bde-ac6b-3541c42fb90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:38.654331 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1084/2164] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:38:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:38:38.659213 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e914a51-1e15-4155-a4b8-013286079b66 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:38.696103 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e914a51-1e15-4155-a4b8-013286079b66 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:38.712355 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bfc526-6f35-4796-80d4-cbbd8a11f2ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:38.713283 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bfc526-6f35-4796-80d4-cbbd8a11f2ca admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:38.713674 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bfc526-6f35-4796-80d4-cbbd8a11f2ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:38.715409 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bfc526-6f35-4796-80d4-cbbd8a11f2ca admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:38.718396 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bfc526-6f35-4796-80d4-cbbd8a11f2ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:38:38.719932 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bfc526-6f35-4796-80d4-cbbd8a11f2ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:38.728543 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bfc526-6f35-4796-80d4-cbbd8a11f2ca admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:38.778228 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1081/2165] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jul 15 19:38:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:38:38.779577 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba67b4b-042d-4bde-ac6b-3541c42fb90f None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:38.793692 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba67b4b-042d-4bde-ac6b-3541c42fb90f None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:38.808805 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5289a68-07cc-47ac-84ea-1fe30e98bc5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:38.809529 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5289a68-07cc-47ac-84ea-1fe30e98bc5f admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:38.820068 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5289a68-07cc-47ac-84ea-1fe30e98bc5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:38.820068 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5289a68-07cc-47ac-84ea-1fe30e98bc5f admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:38.820068 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5289a68-07cc-47ac-84ea-1fe30e98bc5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:38.820068 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5289a68-07cc-47ac-84ea-1fe30e98bc5f admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:39.044271 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1085/2166] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jul 15 19:38:38 2022] POST /identity/v3/projects => generated 385 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:38:39.044509 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bfc526-6f35-4796-80d4-cbbd8a11f2ca admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:39.044509 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bfc526-6f35-4796-80d4-cbbd8a11f2ca admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:39.125462 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9257420a-6c6b-406f-880b-4282c9416805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:39.125462 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9257420a-6c6b-406f-880b-4282c9416805 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:39.125462 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9257420a-6c6b-406f-880b-4282c9416805 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:39.128804 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9257420a-6c6b-406f-880b-4282c9416805 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:39.129661 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9257420a-6c6b-406f-880b-4282c9416805 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:39.133518 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9257420a-6c6b-406f-880b-4282c9416805 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:39.327427 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1082/2167] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jul 15 19:38:39 2022] POST /identity/v3/users => generated 293 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:38:39.333239 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5289a68-07cc-47ac-84ea-1fe30e98bc5f admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:39.346136 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5289a68-07cc-47ac-84ea-1fe30e98bc5f admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:39.351489 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2893a9-30fd-47e7-87f5-41273c20e6bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:39.352549 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2893a9-30fd-47e7-87f5-41273c20e6bd admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:39.353008 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2893a9-30fd-47e7-87f5-41273c20e6bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:39.353297 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2893a9-30fd-47e7-87f5-41273c20e6bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:39.354477 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2893a9-30fd-47e7-87f5-41273c20e6bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:39.356165 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2893a9-30fd-47e7-87f5-41273c20e6bd admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:39.368259 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1086/2168] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:38:39 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:38:39.372774 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9257420a-6c6b-406f-880b-4282c9416805 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:39.389324 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9257420a-6c6b-406f-880b-4282c9416805 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:39.395386 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449f9384-2b36-4d97-933b-a8a30d990441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:39.396302 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449f9384-2b36-4d97-933b-a8a30d990441 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:39.400020 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449f9384-2b36-4d97-933b-a8a30d990441 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:39.400020 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449f9384-2b36-4d97-933b-a8a30d990441 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:39.400020 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449f9384-2b36-4d97-933b-a8a30d990441 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:39.400020 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449f9384-2b36-4d97-933b-a8a30d990441 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:39.413582 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1083/2169] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:38:39 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:38:39.415045 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2893a9-30fd-47e7-87f5-41273c20e6bd admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:39.427111 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2893a9-30fd-47e7-87f5-41273c20e6bd admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:39.434376 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecd17e1-f8b3-45c1-b370-6c69a4f24302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:39.437123 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecd17e1-f8b3-45c1-b370-6c69a4f24302 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:39.437487 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecd17e1-f8b3-45c1-b370-6c69a4f24302 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:39.437860 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecd17e1-f8b3-45c1-b370-6c69a4f24302 admin admin] PATH_INFO: `/v3/projects/02b78ee6c10646b8bef175c253528a63/users/b0f45d14ecea4204bea91708627178bf/roles/28763c6e70094116b3a4b2895895fa58` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:39.460212 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecd17e1-f8b3-45c1-b370-6c69a4f24302 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02b78ee6c10646b8bef175c253528a63, user_id=b0f45d14ecea4204bea91708627178bf, role_id=28763c6e70094116b3a4b2895895fa58)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:39.473870 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecd17e1-f8b3-45c1-b370-6c69a4f24302 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:39.566351 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:38:39.693932 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1087/2170] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:38:39 2022] PUT /identity/v3/projects/02b78ee6c10646b8bef175c253528a63/users/b0f45d14ecea4204bea91708627178bf/roles/28763c6e70094116b3a4b2895895fa58 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:38:39.708370 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a0984-11ee-4291-b2bc-02da67a1ef7e None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:39.708670 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a0984-11ee-4291-b2bc-02da67a1ef7e None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:39.708955 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a0984-11ee-4291-b2bc-02da67a1ef7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:39.759864 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-8e4a0984-11ee-4291-b2bc-02da67a1ef7e None None] MFA Rules not processed for user `b0f45d14ecea4204bea91708627178bf`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:38:39.761121 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-8e4a0984-11ee-4291-b2bc-02da67a1ef7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:39.865346 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1084/2171] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:38:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:38:39.870191 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecd17e1-f8b3-45c1-b370-6c69a4f24302 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:40.004158 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecd17e1-f8b3-45c1-b370-6c69a4f24302 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:40.018574 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befa33c5-8993-4952-bd87-62b9afeda39a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:40.019272 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befa33c5-8993-4952-bd87-62b9afeda39a admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:40.019558 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befa33c5-8993-4952-bd87-62b9afeda39a admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:40.019836 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befa33c5-8993-4952-bd87-62b9afeda39a admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:40.020540 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befa33c5-8993-4952-bd87-62b9afeda39a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:40.026968 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befa33c5-8993-4952-bd87-62b9afeda39a admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:40.158251 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1088/2172] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jul 15 19:38:39 2022] POST /identity/v3/projects => generated 387 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:38:40.162492 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4a0984-11ee-4291-b2bc-02da67a1ef7e None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:40.176652 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:DELETE=7,keystone:SELECT=87,keystone:INSERT=6 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:38:40.182188 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4a0984-11ee-4291-b2bc-02da67a1ef7e None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:40.315172 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f6c883-b98c-49f2-8fb6-db4ad85f106f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:40.316424 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f6c883-b98c-49f2-8fb6-db4ad85f106f admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:40.316859 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f6c883-b98c-49f2-8fb6-db4ad85f106f admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:40.317289 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f6c883-b98c-49f2-8fb6-db4ad85f106f admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:40.318177 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f6c883-b98c-49f2-8fb6-db4ad85f106f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:40.320892 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f6c883-b98c-49f2-8fb6-db4ad85f106f admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:40.498980 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1085/2173] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jul 15 19:38:40 2022] POST /identity/v3/users => generated 300 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:38:40.499412 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befa33c5-8993-4952-bd87-62b9afeda39a admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:40.512436 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befa33c5-8993-4952-bd87-62b9afeda39a admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:40.536459 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899dec6b-30ce-4f59-ba21-45066c4cf15f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:40.537204 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899dec6b-30ce-4f59-ba21-45066c4cf15f admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:40.537493 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899dec6b-30ce-4f59-ba21-45066c4cf15f admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:40.538000 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899dec6b-30ce-4f59-ba21-45066c4cf15f admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:40.538753 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899dec6b-30ce-4f59-ba21-45066c4cf15f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:40.540839 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899dec6b-30ce-4f59-ba21-45066c4cf15f admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:40.566849 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1089/2174] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:38:40 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:38:40.570717 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f6c883-b98c-49f2-8fb6-db4ad85f106f admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:40.586150 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f6c883-b98c-49f2-8fb6-db4ad85f106f admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:40.598857 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9104f842-35aa-403c-8284-1ad0b990c48f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:40.600325 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9104f842-35aa-403c-8284-1ad0b990c48f admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:40.600747 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9104f842-35aa-403c-8284-1ad0b990c48f admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:40.601125 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9104f842-35aa-403c-8284-1ad0b990c48f admin admin] PATH_INFO: `/v3/projects/a56448f2c5af4402bfa75ba867edf876/users/69ff0d7f6b6c4839918632d659f67370/roles/4318f86ba9f74cd4a0bef1c311276657` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:40.624782 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9104f842-35aa-403c-8284-1ad0b990c48f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a56448f2c5af4402bfa75ba867edf876, user_id=69ff0d7f6b6c4839918632d659f67370, role_id=4318f86ba9f74cd4a0bef1c311276657)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:40.628357 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9104f842-35aa-403c-8284-1ad0b990c48f admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:40.794575 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1086/2175] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:38:40 2022] PUT /identity/v3/projects/a56448f2c5af4402bfa75ba867edf876/users/69ff0d7f6b6c4839918632d659f67370/roles/4318f86ba9f74cd4a0bef1c311276657 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:38:40.798835 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899dec6b-30ce-4f59-ba21-45066c4cf15f admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:40.815632 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899dec6b-30ce-4f59-ba21-45066c4cf15f admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:40.891955 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0faea2-7fc4-4a25-9834-1c37f4360f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:40.893619 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0faea2-7fc4-4a25-9834-1c37f4360f34 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:40.894267 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0faea2-7fc4-4a25-9834-1c37f4360f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:40.894779 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0faea2-7fc4-4a25-9834-1c37f4360f34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:40.896243 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0faea2-7fc4-4a25-9834-1c37f4360f34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:40.898255 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0faea2-7fc4-4a25-9834-1c37f4360f34 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:40.918479 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1090/2176] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:38:40 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:38:40.918696 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9104f842-35aa-403c-8284-1ad0b990c48f admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:40.932586 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9104f842-35aa-403c-8284-1ad0b990c48f admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:40.938962 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90be1e85-88de-4a73-8a1a-f161acfebf52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:40.941383 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90be1e85-88de-4a73-8a1a-f161acfebf52 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:40.941648 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90be1e85-88de-4a73-8a1a-f161acfebf52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:40.941954 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90be1e85-88de-4a73-8a1a-f161acfebf52 admin admin] PATH_INFO: `/v3/projects/a56448f2c5af4402bfa75ba867edf876/users/69ff0d7f6b6c4839918632d659f67370/roles/4318f86ba9f74cd4a0bef1c311276657` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:40.944481 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90be1e85-88de-4a73-8a1a-f161acfebf52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a56448f2c5af4402bfa75ba867edf876, user_id=69ff0d7f6b6c4839918632d659f67370, role_id=4318f86ba9f74cd4a0bef1c311276657)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:40.947416 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90be1e85-88de-4a73-8a1a-f161acfebf52 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:40.982094 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1087/2177] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:38:40 2022] PUT /identity/v3/projects/a56448f2c5af4402bfa75ba867edf876/users/69ff0d7f6b6c4839918632d659f67370/roles/4318f86ba9f74cd4a0bef1c311276657 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:38:40.992486 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72000095-f47b-42ca-b7b3-7e13fd8d2984 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:40.994650 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72000095-f47b-42ca-b7b3-7e13fd8d2984 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:40.994935 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72000095-f47b-42ca-b7b3-7e13fd8d2984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:41.056075 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-72000095-f47b-42ca-b7b3-7e13fd8d2984 None None] MFA Rules not processed for user `69ff0d7f6b6c4839918632d659f67370`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:38:41.057340 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-72000095-f47b-42ca-b7b3-7e13fd8d2984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:41.172972 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1091/2178] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:38:40 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:38:41.186523 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90be1e85-88de-4a73-8a1a-f161acfebf52 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:41.189143 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-90be1e85-88de-4a73-8a1a-f161acfebf52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:41.270484 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90be1e85-88de-4a73-8a1a-f161acfebf52 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:41.276362 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911fdd33-331c-4126-a7c1-8afc9071bf86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86c2df3d378490ca7ea497a73c19d40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:41.277565 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911fdd33-331c-4126-a7c1-8afc9071bf86 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:41.278109 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911fdd33-331c-4126-a7c1-8afc9071bf86 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:41.278672 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911fdd33-331c-4126-a7c1-8afc9071bf86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:41.296606 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911fdd33-331c-4126-a7c1-8afc9071bf86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:41.298276 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911fdd33-331c-4126-a7c1-8afc9071bf86 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:41.313181 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1088/2179] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:38:41 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:38:41.355761 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72000095-f47b-42ca-b7b3-7e13fd8d2984 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:41.451290 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72000095-f47b-42ca-b7b3-7e13fd8d2984 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:41.457022 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2245168c-a418-45ef-abc0-15d0dd9c870a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:41.458419 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2245168c-a418-45ef-abc0-15d0dd9c870a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:41.458802 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2245168c-a418-45ef-abc0-15d0dd9c870a admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:41.459158 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2245168c-a418-45ef-abc0-15d0dd9c870a admin admin] PATH_INFO: `/v3/users/b0f45d14ecea4204bea91708627178bf` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:41.460612 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2245168c-a418-45ef-abc0-15d0dd9c870a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0f45d14ecea4204bea91708627178bf)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:41.462739 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2245168c-a418-45ef-abc0-15d0dd9c870a admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:41.857700 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-2245168c-a418-45ef-abc0-15d0dd9c870a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0f45d14ecea4204bea91708627178bf'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:41.882227 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-2245168c-a418-45ef-abc0-15d0dd9c870a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0f45d14ecea4204bea91708627178bf'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:42.002667 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-2245168c-a418-45ef-abc0-15d0dd9c870a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0f45d14ecea4204bea91708627178bf'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:42.003360 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-2245168c-a418-45ef-abc0-15d0dd9c870a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0f45d14ecea4204bea91708627178bf'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:42.004160 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-2245168c-a418-45ef-abc0-15d0dd9c870a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0f45d14ecea4204bea91708627178bf'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:42.020633 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1092/2180] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jul 15 19:38:41 2022] DELETE /identity/v3/users/b0f45d14ecea4204bea91708627178bf => generated 0 bytes in 666 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:38:42.025429 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f9f6f-4a74-4f8d-a1f0-73ac91d29fd0 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:42.025980 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f9f6f-4a74-4f8d-a1f0-73ac91d29fd0 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:42.026341 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f9f6f-4a74-4f8d-a1f0-73ac91d29fd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:42.050473 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-226f9f6f-4a74-4f8d-a1f0-73ac91d29fd0 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:38:42.051828 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-226f9f6f-4a74-4f8d-a1f0-73ac91d29fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:42.148432 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1089/2181] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:38:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:38:42.164174 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2245168c-a418-45ef-abc0-15d0dd9c870a admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:42.169621 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-2245168c-a418-45ef-abc0-15d0dd9c870a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:42.239665 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2245168c-a418-45ef-abc0-15d0dd9c870a admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:42.244269 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaede0fd-aa01-428f-a940-1283b5889e8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:42.244943 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaede0fd-aa01-428f-a940-1283b5889e8c service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:42.245246 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaede0fd-aa01-428f-a940-1283b5889e8c service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:42.245691 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaede0fd-aa01-428f-a940-1283b5889e8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:42.261187 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaede0fd-aa01-428f-a940-1283b5889e8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:42.265800 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaede0fd-aa01-428f-a940-1283b5889e8c service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:42.278254 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1093/2182] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:38:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:38:42.854884 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226f9f6f-4a74-4f8d-a1f0-73ac91d29fd0 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:42.856836 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-226f9f6f-4a74-4f8d-a1f0-73ac91d29fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:42.871848 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226f9f6f-4a74-4f8d-a1f0-73ac91d29fd0 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:42.877431 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5986c27-24f4-481d-a6d1-9c3514730f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:42.879490 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5986c27-24f4-481d-a6d1-9c3514730f3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:42.879960 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5986c27-24f4-481d-a6d1-9c3514730f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:42.880400 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5986c27-24f4-481d-a6d1-9c3514730f3b admin admin] PATH_INFO: `/v3/users/7337f992172c4a4bb1cb778d9cb09284` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:42.882072 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5986c27-24f4-481d-a6d1-9c3514730f3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7337f992172c4a4bb1cb778d9cb09284)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:42.884651 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5986c27-24f4-481d-a6d1-9c3514730f3b admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:43.063273 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaede0fd-aa01-428f-a940-1283b5889e8c service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:43.063970 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-aaede0fd-aa01-428f-a940-1283b5889e8c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:43.088696 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaede0fd-aa01-428f-a940-1283b5889e8c service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:43.094760 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb48b0b-dfdf-4741-a690-3a0424f453d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:43.097136 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb48b0b-dfdf-4741-a690-3a0424f453d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:43.097586 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb48b0b-dfdf-4741-a690-3a0424f453d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:43.098386 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb48b0b-dfdf-4741-a690-3a0424f453d1 admin admin] PATH_INFO: `/v3/projects/02b78ee6c10646b8bef175c253528a63` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:43.099814 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb48b0b-dfdf-4741-a690-3a0424f453d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02b78ee6c10646b8bef175c253528a63)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:43.102359 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb48b0b-dfdf-4741-a690-3a0424f453d1 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:43.188312 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d5986c27-24f4-481d-a6d1-9c3514730f3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7337f992172c4a4bb1cb778d9cb09284'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:43.201348 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d5986c27-24f4-481d-a6d1-9c3514730f3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7337f992172c4a4bb1cb778d9cb09284'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:43.223296 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-0fb48b0b-dfdf-4741-a690-3a0424f453d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02b78ee6c10646b8bef175c253528a63', 'request_id': 'req-0fb48b0b-dfdf-4741-a690-3a0424f453d1'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:43.291893 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d5986c27-24f4-481d-a6d1-9c3514730f3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7337f992172c4a4bb1cb778d9cb09284'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:43.292520 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d5986c27-24f4-481d-a6d1-9c3514730f3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7337f992172c4a4bb1cb778d9cb09284'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:43.301421 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d5986c27-24f4-481d-a6d1-9c3514730f3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7337f992172c4a4bb1cb778d9cb09284'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:43.310115 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1090/2183] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jul 15 19:38:42 2022] DELETE /identity/v3/users/7337f992172c4a4bb1cb778d9cb09284 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:38:43.315089 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab591ae-9714-436d-b63f-669eb2850963 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:43.315528 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab591ae-9714-436d-b63f-669eb2850963 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:43.315809 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab591ae-9714-436d-b63f-669eb2850963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:43.334107 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-7ab591ae-9714-436d-b63f-669eb2850963 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:38:43.335329 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-7ab591ae-9714-436d-b63f-669eb2850963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:43.396355 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-0fb48b0b-dfdf-4741-a690-3a0424f453d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02b78ee6c10646b8bef175c253528a63', 'request_id': 'req-0fb48b0b-dfdf-4741-a690-3a0424f453d1'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:43.404134 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-0fb48b0b-dfdf-4741-a690-3a0424f453d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02b78ee6c10646b8bef175c253528a63', 'request_id': 'req-0fb48b0b-dfdf-4741-a690-3a0424f453d1'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:43.410842 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1091/2184] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:38:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:38:43.420945 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab591ae-9714-436d-b63f-669eb2850963 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:43.422667 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-7ab591ae-9714-436d-b63f-669eb2850963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:43.426809 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-0fb48b0b-dfdf-4741-a690-3a0424f453d1 admin admin] The token cache is being invalidate because project 02b78ee6c10646b8bef175c253528a63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:38:43.427115 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-0fb48b0b-dfdf-4741-a690-3a0424f453d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:43.427773 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-0fb48b0b-dfdf-4741-a690-3a0424f453d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:43.429637 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1094/2185] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jul 15 19:38:43 2022] DELETE /identity/v3/projects/02b78ee6c10646b8bef175c253528a63 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:38:43.435544 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb48b0b-dfdf-4741-a690-3a0424f453d1 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:43.437544 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-0fb48b0b-dfdf-4741-a690-3a0424f453d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:43.498666 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab591ae-9714-436d-b63f-669eb2850963 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:43.503486 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed92216-aae0-401c-a970-b871c171491c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:43.504633 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed92216-aae0-401c-a970-b871c171491c service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:43.504741 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed92216-aae0-401c-a970-b871c171491c service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:43.505065 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed92216-aae0-401c-a970-b871c171491c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:43.511942 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-5ed92216-aae0-401c-a970-b871c171491c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:43.526305 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed92216-aae0-401c-a970-b871c171491c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:43.527885 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed92216-aae0-401c-a970-b871c171491c service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:43.535475 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb48b0b-dfdf-4741-a690-3a0424f453d1 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:43.539692 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1092/2186] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:38:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 19:38:43.543498 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64da0720-ffbf-428c-8688-b23147917e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:43.544515 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da0720-ffbf-428c-8688-b23147917e25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:43.544999 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da0720-ffbf-428c-8688-b23147917e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:43.545280 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da0720-ffbf-428c-8688-b23147917e25 admin admin] PATH_INFO: `/v3/users/69ff0d7f6b6c4839918632d659f67370` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:43.547063 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64da0720-ffbf-428c-8688-b23147917e25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69ff0d7f6b6c4839918632d659f67370)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:43.549060 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64da0720-ffbf-428c-8688-b23147917e25 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:43.946291 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-64da0720-ffbf-428c-8688-b23147917e25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69ff0d7f6b6c4839918632d659f67370'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:43.947585 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:38:43.971617 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-64da0720-ffbf-428c-8688-b23147917e25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69ff0d7f6b6c4839918632d659f67370'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:44.079708 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-64da0720-ffbf-428c-8688-b23147917e25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69ff0d7f6b6c4839918632d659f67370'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:44.082802 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-64da0720-ffbf-428c-8688-b23147917e25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69ff0d7f6b6c4839918632d659f67370'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:44.088084 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-64da0720-ffbf-428c-8688-b23147917e25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69ff0d7f6b6c4839918632d659f67370'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:44.108711 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1095/2187] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jul 15 19:38:43 2022] DELETE /identity/v3/users/69ff0d7f6b6c4839918632d659f67370 => generated 0 bytes in 674 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:38:44.272779 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:38:44.320967 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed92216-aae0-401c-a970-b871c171491c service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:44.322636 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-5ed92216-aae0-401c-a970-b871c171491c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:44.335774 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed92216-aae0-401c-a970-b871c171491c service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:44.397450 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2b8993-508e-42d9-a184-5b1bf731bd9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:44.398410 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2b8993-508e-42d9-a184-5b1bf731bd9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:44.398697 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2b8993-508e-42d9-a184-5b1bf731bd9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:44.398972 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2b8993-508e-42d9-a184-5b1bf731bd9a admin admin] PATH_INFO: `/v3/projects/e2e1abaaf77c443096c57263c1451cb2` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:44.400205 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2b8993-508e-42d9-a184-5b1bf731bd9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2e1abaaf77c443096c57263c1451cb2)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:44.402384 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2b8993-508e-42d9-a184-5b1bf731bd9a admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:44.576017 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6b2b8993-508e-42d9-a184-5b1bf731bd9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2e1abaaf77c443096c57263c1451cb2', 'request_id': 'req-6b2b8993-508e-42d9-a184-5b1bf731bd9a'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:44.749796 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6b2b8993-508e-42d9-a184-5b1bf731bd9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2e1abaaf77c443096c57263c1451cb2', 'request_id': 'req-6b2b8993-508e-42d9-a184-5b1bf731bd9a'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:44.759457 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6b2b8993-508e-42d9-a184-5b1bf731bd9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2e1abaaf77c443096c57263c1451cb2', 'request_id': 'req-6b2b8993-508e-42d9-a184-5b1bf731bd9a'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:44.781349 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6b2b8993-508e-42d9-a184-5b1bf731bd9a admin admin] The token cache is being invalidate because project e2e1abaaf77c443096c57263c1451cb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:38:44.781349 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6b2b8993-508e-42d9-a184-5b1bf731bd9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:44.781349 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6b2b8993-508e-42d9-a184-5b1bf731bd9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:44.781349 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1093/2188] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jul 15 19:38:44 2022] DELETE /identity/v3/projects/e2e1abaaf77c443096c57263c1451cb2 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:38:44.782068 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64da0720-ffbf-428c-8688-b23147917e25 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:44.783337 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-64da0720-ffbf-428c-8688-b23147917e25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:44.797243 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64da0720-ffbf-428c-8688-b23147917e25 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:44.878487 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4803e8d-45e9-4adb-a32a-37094f6edf07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:44.879489 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4803e8d-45e9-4adb-a32a-37094f6edf07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:44.879881 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4803e8d-45e9-4adb-a32a-37094f6edf07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:44.880298 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4803e8d-45e9-4adb-a32a-37094f6edf07 admin admin] PATH_INFO: `/v3/users/cdfa7019c4054a48a4f0fdb4bc6ef2e6` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:44.882495 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4803e8d-45e9-4adb-a32a-37094f6edf07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdfa7019c4054a48a4f0fdb4bc6ef2e6)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:44.884401 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4803e8d-45e9-4adb-a32a-37094f6edf07 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:44.932382 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2b8993-508e-42d9-a184-5b1bf731bd9a admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:44.934329 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-6b2b8993-508e-42d9-a184-5b1bf731bd9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:44.957330 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2b8993-508e-42d9-a184-5b1bf731bd9a admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:44.964859 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11a6a4e-4f18-482b-a80f-a399f887cd92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:44.964859 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11a6a4e-4f18-482b-a80f-a399f887cd92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:44.964859 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11a6a4e-4f18-482b-a80f-a399f887cd92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:44.964859 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11a6a4e-4f18-482b-a80f-a399f887cd92 admin admin] PATH_INFO: `/v3/projects/a56448f2c5af4402bfa75ba867edf876` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:44.966395 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11a6a4e-4f18-482b-a80f-a399f887cd92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a56448f2c5af4402bfa75ba867edf876)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:44.968566 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11a6a4e-4f18-482b-a80f-a399f887cd92 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:45.133359 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-a4803e8d-45e9-4adb-a32a-37094f6edf07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdfa7019c4054a48a4f0fdb4bc6ef2e6'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:45.138995 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-c11a6a4e-4f18-482b-a80f-a399f887cd92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a56448f2c5af4402bfa75ba867edf876', 'request_id': 'req-c11a6a4e-4f18-482b-a80f-a399f887cd92'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:45.149634 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-a4803e8d-45e9-4adb-a32a-37094f6edf07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdfa7019c4054a48a4f0fdb4bc6ef2e6'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:45.237125 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-c11a6a4e-4f18-482b-a80f-a399f887cd92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a56448f2c5af4402bfa75ba867edf876', 'request_id': 'req-c11a6a4e-4f18-482b-a80f-a399f887cd92'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:45.247652 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-c11a6a4e-4f18-482b-a80f-a399f887cd92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a56448f2c5af4402bfa75ba867edf876', 'request_id': 'req-c11a6a4e-4f18-482b-a80f-a399f887cd92'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:45.261501 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-c11a6a4e-4f18-482b-a80f-a399f887cd92 admin admin] The token cache is being invalidate because project a56448f2c5af4402bfa75ba867edf876 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:38:45.262000 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-c11a6a4e-4f18-482b-a80f-a399f887cd92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:45.263074 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-c11a6a4e-4f18-482b-a80f-a399f887cd92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:45.269168 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1094/2189] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jul 15 19:38:44 2022] DELETE /identity/v3/projects/a56448f2c5af4402bfa75ba867edf876 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:38:45.275225 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e6daf-445d-4d55-9442-5f6080ffb3e1 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:45.275845 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e6daf-445d-4d55-9442-5f6080ffb3e1 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:45.276044 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e6daf-445d-4d55-9442-5f6080ffb3e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:45.303315 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-080e6daf-445d-4d55-9442-5f6080ffb3e1 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:38:45.305119 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-080e6daf-445d-4d55-9442-5f6080ffb3e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:45.344248 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-a4803e8d-45e9-4adb-a32a-37094f6edf07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdfa7019c4054a48a4f0fdb4bc6ef2e6'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:45.345140 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-a4803e8d-45e9-4adb-a32a-37094f6edf07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdfa7019c4054a48a4f0fdb4bc6ef2e6'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:45.346066 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-a4803e8d-45e9-4adb-a32a-37094f6edf07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdfa7019c4054a48a4f0fdb4bc6ef2e6'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:45.370875 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1096/2190] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jul 15 19:38:44 2022] DELETE /identity/v3/users/cdfa7019c4054a48a4f0fdb4bc6ef2e6 => generated 0 bytes in 590 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:38:45.398924 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1095/2191] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:38:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:38:45.408256 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab03c3d-cc0f-4bd3-9043-2e79e5fac7ca None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:45.408880 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab03c3d-cc0f-4bd3-9043-2e79e5fac7ca None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:45.409307 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab03c3d-cc0f-4bd3-9043-2e79e5fac7ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:45.434925 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-dab03c3d-cc0f-4bd3-9043-2e79e5fac7ca None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:38:45.436994 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-dab03c3d-cc0f-4bd3-9043-2e79e5fac7ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:45.519727 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1097/2192] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:38:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:38:45.523865 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080e6daf-445d-4d55-9442-5f6080ffb3e1 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:45.538905 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080e6daf-445d-4d55-9442-5f6080ffb3e1 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:45.553598 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cc93e5-517f-47e8-8d31-695124f673cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:45.554315 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cc93e5-517f-47e8-8d31-695124f673cd admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:45.554597 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cc93e5-517f-47e8-8d31-695124f673cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:45.554870 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cc93e5-517f-47e8-8d31-695124f673cd admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:45.556012 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cc93e5-517f-47e8-8d31-695124f673cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:38:45.556327 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cc93e5-517f-47e8-8d31-695124f673cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:45.557974 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cc93e5-517f-47e8-8d31-695124f673cd admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:45.585160 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1096/2193] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jul 15 19:38:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:38:45.588777 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab03c3d-cc0f-4bd3-9043-2e79e5fac7ca None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:45.603062 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab03c3d-cc0f-4bd3-9043-2e79e5fac7ca None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:45.615129 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec9a8f9-11ae-4c10-b027-20901ce2a1d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:45.615822 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec9a8f9-11ae-4c10-b027-20901ce2a1d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:45.616466 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec9a8f9-11ae-4c10-b027-20901ce2a1d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:45.616768 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec9a8f9-11ae-4c10-b027-20901ce2a1d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:45.617786 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec9a8f9-11ae-4c10-b027-20901ce2a1d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:45.622162 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec9a8f9-11ae-4c10-b027-20901ce2a1d2 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:45.792671 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1098/2194] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jul 15 19:38:45 2022] POST /identity/v3/projects => generated 385 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:38:45.797081 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cc93e5-517f-47e8-8d31-695124f673cd admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:45.827290 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cc93e5-517f-47e8-8d31-695124f673cd admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:45.922639 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f8c26a-d428-4b65-9701-c5a409e87621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:45.923345 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f8c26a-d428-4b65-9701-c5a409e87621 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:45.923662 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f8c26a-d428-4b65-9701-c5a409e87621 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:45.924465 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f8c26a-d428-4b65-9701-c5a409e87621 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:45.925709 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f8c26a-d428-4b65-9701-c5a409e87621 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:45.928147 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f8c26a-d428-4b65-9701-c5a409e87621 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:46.373720 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be35dac-014b-4969-8529-30785511616d None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:46.374354 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be35dac-014b-4969-8529-30785511616d None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:46.374354 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be35dac-014b-4969-8529-30785511616d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:46.407255 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-8be35dac-014b-4969-8529-30785511616d None None] MFA Rules not processed for user `f8fed2fa41894eaeafa009161d8bc537`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:38:46.409308 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-8be35dac-014b-4969-8529-30785511616d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:46.479644 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1097/2195] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jul 15 19:38:45 2022] POST /identity/v3/users => generated 293 bytes in 680 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:38:46.483357 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f8c26a-d428-4b65-9701-c5a409e87621 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:46.498093 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1099/2196] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:38:46 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:38:46.499723 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f8c26a-d428-4b65-9701-c5a409e87621 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:46.506425 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397c4d14-07b6-4dc2-8fa7-a6d71f1c8450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:46.506866 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be35dac-014b-4969-8529-30785511616d None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:46.508213 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397c4d14-07b6-4dc2-8fa7-a6d71f1c8450 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:46.508724 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397c4d14-07b6-4dc2-8fa7-a6d71f1c8450 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:46.509173 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397c4d14-07b6-4dc2-8fa7-a6d71f1c8450 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:46.510119 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397c4d14-07b6-4dc2-8fa7-a6d71f1c8450 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:46.512183 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397c4d14-07b6-4dc2-8fa7-a6d71f1c8450 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:46.517022 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-8be35dac-014b-4969-8529-30785511616d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:46.563168 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1098/2197] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:38:46 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:38:46.568887 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397c4d14-07b6-4dc2-8fa7-a6d71f1c8450 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:46.574050 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-397c4d14-07b6-4dc2-8fa7-a6d71f1c8450 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:46.597853 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397c4d14-07b6-4dc2-8fa7-a6d71f1c8450 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:46.603812 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8ef8b5-cacf-496d-b583-299cdfab4f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:46.606318 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8ef8b5-cacf-496d-b583-299cdfab4f59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:46.606318 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8ef8b5-cacf-496d-b583-299cdfab4f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:46.606318 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8ef8b5-cacf-496d-b583-299cdfab4f59 admin admin] PATH_INFO: `/v3/projects/f311050b5ed44425a4856a5c2aa3360d` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:46.606720 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8ef8b5-cacf-496d-b583-299cdfab4f59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f311050b5ed44425a4856a5c2aa3360d)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:46.620260 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8ef8b5-cacf-496d-b583-299cdfab4f59 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:46.646413 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be35dac-014b-4969-8529-30785511616d None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:46.658264 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dceb41-f35a-482d-8f4f-aa9152e7e0e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7673a3b5c0324e07a9535ac70ed7a491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:46.662417 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dceb41-f35a-482d-8f4f-aa9152e7e0e3 service nova] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:46.662417 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dceb41-f35a-482d-8f4f-aa9152e7e0e3 service nova] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:46.662417 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dceb41-f35a-482d-8f4f-aa9152e7e0e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:46.665127 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-01dceb41-f35a-482d-8f4f-aa9152e7e0e3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:46.744481 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dceb41-f35a-482d-8f4f-aa9152e7e0e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:46.746295 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dceb41-f35a-482d-8f4f-aa9152e7e0e3 service nova] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:46.757673 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1100/2198] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:38:46 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:38:46.758864 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dceb41-f35a-482d-8f4f-aa9152e7e0e3 service nova] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:46.769215 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dceb41-f35a-482d-8f4f-aa9152e7e0e3 service nova] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:46.778643 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32896cb-53ff-4b89-b796-31b6a3838c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:46.778643 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32896cb-53ff-4b89-b796-31b6a3838c73 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:46.778643 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32896cb-53ff-4b89-b796-31b6a3838c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:46.778643 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32896cb-53ff-4b89-b796-31b6a3838c73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:46.783516 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32896cb-53ff-4b89-b796-31b6a3838c73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:46.783516 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32896cb-53ff-4b89-b796-31b6a3838c73 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:46.795497 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1101/2199] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:38:46 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:38:46.801027 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32896cb-53ff-4b89-b796-31b6a3838c73 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:46.806701 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-cf8ef8b5-cacf-496d-b583-299cdfab4f59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f311050b5ed44425a4856a5c2aa3360d', 'request_id': 'req-cf8ef8b5-cacf-496d-b583-299cdfab4f59'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:46.814486 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32896cb-53ff-4b89-b796-31b6a3838c73 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:46.883493 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f75806-b9d5-4adc-a8d6-a5f6a601950e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:46.884534 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f75806-b9d5-4adc-a8d6-a5f6a601950e admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:46.884920 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f75806-b9d5-4adc-a8d6-a5f6a601950e admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:46.885524 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f75806-b9d5-4adc-a8d6-a5f6a601950e admin admin] PATH_INFO: `/v3/projects/3c5efd83a79f410b8ecd163af9e19cff/users/cb1a604bb9b24a5699d3bad4d369b654/roles/28763c6e70094116b3a4b2895895fa58` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:46.903371 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-cf8ef8b5-cacf-496d-b583-299cdfab4f59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f311050b5ed44425a4856a5c2aa3360d', 'request_id': 'req-cf8ef8b5-cacf-496d-b583-299cdfab4f59'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:46.919981 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-cf8ef8b5-cacf-496d-b583-299cdfab4f59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f311050b5ed44425a4856a5c2aa3360d', 'request_id': 'req-cf8ef8b5-cacf-496d-b583-299cdfab4f59'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:46.938430 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-cf8ef8b5-cacf-496d-b583-299cdfab4f59 admin admin] The token cache is being invalidate because project f311050b5ed44425a4856a5c2aa3360d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:38:46.944364 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-cf8ef8b5-cacf-496d-b583-299cdfab4f59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:46.944511 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f75806-b9d5-4adc-a8d6-a5f6a601950e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c5efd83a79f410b8ecd163af9e19cff, user_id=cb1a604bb9b24a5699d3bad4d369b654, role_id=28763c6e70094116b3a4b2895895fa58)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:46.944511 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f75806-b9d5-4adc-a8d6-a5f6a601950e admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:46.945109 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-cf8ef8b5-cacf-496d-b583-299cdfab4f59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:38:46.947642 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1099/2200] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jul 15 19:38:46 2022] DELETE /identity/v3/projects/f311050b5ed44425a4856a5c2aa3360d => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:38:46.956787 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e85b0b7-6fed-4860-99da-89a707fa558d None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:46.956896 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e85b0b7-6fed-4860-99da-89a707fa558d None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:46.957162 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e85b0b7-6fed-4860-99da-89a707fa558d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:46.977535 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-2e85b0b7-6fed-4860-99da-89a707fa558d None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:38:46.979632 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-2e85b0b7-6fed-4860-99da-89a707fa558d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:46.989642 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1100/2201] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:38:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:38:46.998327 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffbafa2-7ed6-4087-a508-a53d9ebc6114 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:46.998722 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffbafa2-7ed6-4087-a508-a53d9ebc6114 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:46.999122 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffbafa2-7ed6-4087-a508-a53d9ebc6114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:47.018601 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-5ffbafa2-7ed6-4087-a508-a53d9ebc6114 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:38:47.019791 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-5ffbafa2-7ed6-4087-a508-a53d9ebc6114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:47.028039 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1101/2202] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:38:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:38:47.032251 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffbafa2-7ed6-4087-a508-a53d9ebc6114 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:47.041200 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffbafa2-7ed6-4087-a508-a53d9ebc6114 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:47.045841 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab067d6-239e-46fe-8f39-761bafdfd153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:47.046475 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab067d6-239e-46fe-8f39-761bafdfd153 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:47.046755 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab067d6-239e-46fe-8f39-761bafdfd153 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:47.047057 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab067d6-239e-46fe-8f39-761bafdfd153 admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:47.047795 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab067d6-239e-46fe-8f39-761bafdfd153 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:38:47.048103 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab067d6-239e-46fe-8f39-761bafdfd153 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:47.050252 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab067d6-239e-46fe-8f39-761bafdfd153 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:47.056581 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1102/2203] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:38:46 2022] PUT /identity/v3/projects/3c5efd83a79f410b8ecd163af9e19cff/users/cb1a604bb9b24a5699d3bad4d369b654/roles/28763c6e70094116b3a4b2895895fa58 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:38:47.065195 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1102/2204] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jul 15 19:38:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:38:47.068520 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f75806-b9d5-4adc-a8d6-a5f6a601950e admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:47.074575 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa3c9b5-a690-494e-b2d8-7bcd41ed14c4 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:47.074864 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa3c9b5-a690-494e-b2d8-7bcd41ed14c4 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:47.075146 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa3c9b5-a690-494e-b2d8-7bcd41ed14c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:47.101590 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-5fa3c9b5-a690-494e-b2d8-7bcd41ed14c4 None None] MFA Rules not processed for user `cb1a604bb9b24a5699d3bad4d369b654`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:38:47.102845 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-5fa3c9b5-a690-494e-b2d8-7bcd41ed14c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:47.151824 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f75806-b9d5-4adc-a8d6-a5f6a601950e admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:47.157478 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544786dd-1e36-4bf4-b894-576226fe97af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:47.158800 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544786dd-1e36-4bf4-b894-576226fe97af admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:47.159088 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544786dd-1e36-4bf4-b894-576226fe97af admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:47.159361 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544786dd-1e36-4bf4-b894-576226fe97af admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:47.160059 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544786dd-1e36-4bf4-b894-576226fe97af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:47.162199 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544786dd-1e36-4bf4-b894-576226fe97af admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:47.189914 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1103/2205] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:38:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:38:47.196191 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa3c9b5-a690-494e-b2d8-7bcd41ed14c4 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:47.198351 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-5fa3c9b5-a690-494e-b2d8-7bcd41ed14c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:47.219111 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa3c9b5-a690-494e-b2d8-7bcd41ed14c4 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:47.224651 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421ebce2-7229-469e-98a3-57fb2d575c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:47.225417 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421ebce2-7229-469e-98a3-57fb2d575c23 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:47.225703 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421ebce2-7229-469e-98a3-57fb2d575c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:47.226014 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421ebce2-7229-469e-98a3-57fb2d575c23 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:47.226738 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421ebce2-7229-469e-98a3-57fb2d575c23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:47.229630 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421ebce2-7229-469e-98a3-57fb2d575c23 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:47.237096 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:38:47.271202 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1103/2206] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jul 15 19:38:47 2022] POST /identity/v3/projects => generated 369 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:38:47.273907 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544786dd-1e36-4bf4-b894-576226fe97af admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:47.348188 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544786dd-1e36-4bf4-b894-576226fe97af admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:47.353236 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edf438a-8b02-4093-b154-b28b1f5da7e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:47.353967 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf438a-8b02-4093-b154-b28b1f5da7e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:47.358511 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf438a-8b02-4093-b154-b28b1f5da7e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:47.358511 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf438a-8b02-4093-b154-b28b1f5da7e2 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:47.358511 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edf438a-8b02-4093-b154-b28b1f5da7e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:47.358511 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edf438a-8b02-4093-b154-b28b1f5da7e2 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:47.362317 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1104/2207] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jul 15 19:38:47 2022] POST /identity/v3/projects => generated 387 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:38:47.366214 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421ebce2-7229-469e-98a3-57fb2d575c23 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:47.437637 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421ebce2-7229-469e-98a3-57fb2d575c23 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:47.443018 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e142a83-a338-4773-b23d-f0d904817318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:47.443689 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e142a83-a338-4773-b23d-f0d904817318 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:47.443974 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e142a83-a338-4773-b23d-f0d904817318 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:47.444274 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e142a83-a338-4773-b23d-f0d904817318 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:47.444969 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e142a83-a338-4773-b23d-f0d904817318 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:47.446927 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e142a83-a338-4773-b23d-f0d904817318 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:47.481038 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1104/2208] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jul 15 19:38:47 2022] POST /identity/v3/users => generated 285 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:38:47.484933 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edf438a-8b02-4093-b154-b28b1f5da7e2 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:47.494335 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edf438a-8b02-4093-b154-b28b1f5da7e2 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:47.499289 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3f4760-9483-4b1f-a8ef-1405de4a2f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:47.499949 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f4760-9483-4b1f-a8ef-1405de4a2f07 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:47.502338 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f4760-9483-4b1f-a8ef-1405de4a2f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:47.502338 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f4760-9483-4b1f-a8ef-1405de4a2f07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:47.502338 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3f4760-9483-4b1f-a8ef-1405de4a2f07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:47.503483 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3f4760-9483-4b1f-a8ef-1405de4a2f07 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:47.514092 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1105/2209] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:38:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 19:38:47.518703 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3f4760-9483-4b1f-a8ef-1405de4a2f07 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:47.529397 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3f4760-9483-4b1f-a8ef-1405de4a2f07 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:47.534445 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea4a9bb-57f6-4dcc-98a5-2b8bd1db7c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:47.535178 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea4a9bb-57f6-4dcc-98a5-2b8bd1db7c37 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:47.535469 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea4a9bb-57f6-4dcc-98a5-2b8bd1db7c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:47.535801 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea4a9bb-57f6-4dcc-98a5-2b8bd1db7c37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:47.536588 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea4a9bb-57f6-4dcc-98a5-2b8bd1db7c37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:47.538324 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea4a9bb-57f6-4dcc-98a5-2b8bd1db7c37 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:47.550605 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1106/2210] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:38:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 19:38:47.554904 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea4a9bb-57f6-4dcc-98a5-2b8bd1db7c37 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:47.566813 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea4a9bb-57f6-4dcc-98a5-2b8bd1db7c37 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:47.571971 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bb6b11-bcba-49be-a6a1-56c547909aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:47.573010 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb6b11-bcba-49be-a6a1-56c547909aec admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:47.573289 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb6b11-bcba-49be-a6a1-56c547909aec admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:47.573555 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb6b11-bcba-49be-a6a1-56c547909aec admin admin] PATH_INFO: `/v3/projects/5c906616f9b044559e1c6db9015b97e1/users/e8596191b01c45de99a09e400c3677cb/roles/28763c6e70094116b3a4b2895895fa58` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:47.591866 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bb6b11-bcba-49be-a6a1-56c547909aec admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c906616f9b044559e1c6db9015b97e1, user_id=e8596191b01c45de99a09e400c3677cb, role_id=28763c6e70094116b3a4b2895895fa58)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:47.595910 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bb6b11-bcba-49be-a6a1-56c547909aec admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:47.672606 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1105/2211] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jul 15 19:38:47 2022] POST /identity/v3/users => generated 294 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:38:47.675289 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e142a83-a338-4773-b23d-f0d904817318 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:47.687591 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e142a83-a338-4773-b23d-f0d904817318 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:47.695907 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7ef747-7b47-4430-a6cd-51a638944152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:47.700903 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ef747-7b47-4430-a6cd-51a638944152 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:47.700903 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ef747-7b47-4430-a6cd-51a638944152 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:47.700903 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ef747-7b47-4430-a6cd-51a638944152 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:47.700903 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7ef747-7b47-4430-a6cd-51a638944152 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:47.700903 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7ef747-7b47-4430-a6cd-51a638944152 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:47.719229 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1106/2212] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:38:47 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 19:38:47.720478 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7ef747-7b47-4430-a6cd-51a638944152 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:47.730381 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7ef747-7b47-4430-a6cd-51a638944152 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:47.739188 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1107/2213] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:38:47 2022] PUT /identity/v3/projects/5c906616f9b044559e1c6db9015b97e1/users/e8596191b01c45de99a09e400c3677cb/roles/28763c6e70094116b3a4b2895895fa58 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:38:47.741944 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a712d2-90c8-4507-8e0a-6ca376e4b971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:47.742161 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a712d2-90c8-4507-8e0a-6ca376e4b971 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:47.742442 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a712d2-90c8-4507-8e0a-6ca376e4b971 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:47.742713 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a712d2-90c8-4507-8e0a-6ca376e4b971 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:47.743442 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a712d2-90c8-4507-8e0a-6ca376e4b971 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:47.757683 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10aa5ea-6ef9-483a-9d8f-91b6dd1cf4e2 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:47.757999 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10aa5ea-6ef9-483a-9d8f-91b6dd1cf4e2 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:47.758277 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10aa5ea-6ef9-483a-9d8f-91b6dd1cf4e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:47.761510 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a712d2-90c8-4507-8e0a-6ca376e4b971 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:47.790952 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1107/2214] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:38:47 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:38:47.799748 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a712d2-90c8-4507-8e0a-6ca376e4b971 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:47.804079 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-e10aa5ea-6ef9-483a-9d8f-91b6dd1cf4e2 None None] MFA Rules not processed for user `e8596191b01c45de99a09e400c3677cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:38:47.804558 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-e10aa5ea-6ef9-483a-9d8f-91b6dd1cf4e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:47.827908 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a712d2-90c8-4507-8e0a-6ca376e4b971 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:47.883060 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:38:47.897714 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1108/2215] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:38:47 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:38:47.911101 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10aa5ea-6ef9-483a-9d8f-91b6dd1cf4e2 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:47.912545 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-e10aa5ea-6ef9-483a-9d8f-91b6dd1cf4e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:47.946776 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e4c1a7-bf62-4c67-b798-f53aed4f4343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:47.947715 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e4c1a7-bf62-4c67-b798-f53aed4f4343 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:47.948055 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e4c1a7-bf62-4c67-b798-f53aed4f4343 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:47.948410 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e4c1a7-bf62-4c67-b798-f53aed4f4343 admin admin] PATH_INFO: `/v3/projects/723bd820fa9e40159e90c0650a7c4516/users/da37136a00e34742ab85d94315938fbc/roles/28763c6e70094116b3a4b2895895fa58` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:47.967960 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e4c1a7-bf62-4c67-b798-f53aed4f4343 admin admin] RBAC: Authorizing `identity:create_grant(project_id=723bd820fa9e40159e90c0650a7c4516, user_id=da37136a00e34742ab85d94315938fbc, role_id=28763c6e70094116b3a4b2895895fa58)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:47.972989 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e4c1a7-bf62-4c67-b798-f53aed4f4343 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:48.017860 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10aa5ea-6ef9-483a-9d8f-91b6dd1cf4e2 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:48.021782 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414caa00-e25a-4662-94e8-e6c4935e8080 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04b5b06b0015450a95a28bbcc757cc8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:48.032390 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414caa00-e25a-4662-94e8-e6c4935e8080 service glance] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:48.032390 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414caa00-e25a-4662-94e8-e6c4935e8080 service glance] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:48.032390 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414caa00-e25a-4662-94e8-e6c4935e8080 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:48.032390 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414caa00-e25a-4662-94e8-e6c4935e8080 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:48.034405 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414caa00-e25a-4662-94e8-e6c4935e8080 service glance] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:48.043825 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1109/2216] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:38:47 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 19:38:48.117017 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1108/2217] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:38:47 2022] PUT /identity/v3/projects/723bd820fa9e40159e90c0650a7c4516/users/da37136a00e34742ab85d94315938fbc/roles/28763c6e70094116b3a4b2895895fa58 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:38:48.130033 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0444f6cb-63ce-4507-b576-a842d0cabead None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:48.130576 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0444f6cb-63ce-4507-b576-a842d0cabead None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:48.131247 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0444f6cb-63ce-4507-b576-a842d0cabead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:48.178527 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-0444f6cb-63ce-4507-b576-a842d0cabead None None] MFA Rules not processed for user `da37136a00e34742ab85d94315938fbc`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:38:48.179875 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-0444f6cb-63ce-4507-b576-a842d0cabead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:48.263331 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1110/2218] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:38:48 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:38:48.269519 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e4c1a7-bf62-4c67-b798-f53aed4f4343 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:48.388537 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e4c1a7-bf62-4c67-b798-f53aed4f4343 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:48.400214 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e38888-27b8-480f-b5e6-33b53caa3551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:48.401221 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e38888-27b8-480f-b5e6-33b53caa3551 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:48.403352 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e38888-27b8-480f-b5e6-33b53caa3551 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:48.404254 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e38888-27b8-480f-b5e6-33b53caa3551 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:48.404994 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e38888-27b8-480f-b5e6-33b53caa3551 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:48.408676 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e38888-27b8-480f-b5e6-33b53caa3551 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:48.530378 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1109/2219] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jul 15 19:38:48 2022] POST /identity/v3/projects => generated 387 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:38:48.534540 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0444f6cb-63ce-4507-b576-a842d0cabead None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:48.546765 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0444f6cb-63ce-4507-b576-a842d0cabead None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:48.636638 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025689e6-3779-46cb-8640-0b36de76bb28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:48.637743 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025689e6-3779-46cb-8640-0b36de76bb28 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:48.638417 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025689e6-3779-46cb-8640-0b36de76bb28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:48.638870 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025689e6-3779-46cb-8640-0b36de76bb28 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:48.639731 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025689e6-3779-46cb-8640-0b36de76bb28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:48.641854 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025689e6-3779-46cb-8640-0b36de76bb28 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:48.755822 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1111/2220] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jul 15 19:38:48 2022] POST /identity/v3/users => generated 300 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:38:48.760354 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e38888-27b8-480f-b5e6-33b53caa3551 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:48.770805 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e38888-27b8-480f-b5e6-33b53caa3551 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:48.775415 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ddef06-f18c-476c-baa4-ec0f78e1f59c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:48.776080 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ddef06-f18c-476c-baa4-ec0f78e1f59c admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:48.776567 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ddef06-f18c-476c-baa4-ec0f78e1f59c admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:48.777817 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ddef06-f18c-476c-baa4-ec0f78e1f59c admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:48.779049 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ddef06-f18c-476c-baa4-ec0f78e1f59c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:48.780953 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ddef06-f18c-476c-baa4-ec0f78e1f59c admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:48.796173 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1110/2221] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:38:48 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:38:48.809692 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025689e6-3779-46cb-8640-0b36de76bb28 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:48.822810 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025689e6-3779-46cb-8640-0b36de76bb28 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:48.830302 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75be62b7-e6ec-4038-ad4b-2d9cc3dfd0ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:48.830302 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75be62b7-e6ec-4038-ad4b-2d9cc3dfd0ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:48.830302 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75be62b7-e6ec-4038-ad4b-2d9cc3dfd0ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:48.830302 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75be62b7-e6ec-4038-ad4b-2d9cc3dfd0ee admin admin] PATH_INFO: `/v3/projects/75cd546fbdcb4d538a2284cca8c4637e/users/fa29d89f711341b3922810681eb564da/roles/4318f86ba9f74cd4a0bef1c311276657` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:48.847214 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75be62b7-e6ec-4038-ad4b-2d9cc3dfd0ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=75cd546fbdcb4d538a2284cca8c4637e, user_id=fa29d89f711341b3922810681eb564da, role_id=4318f86ba9f74cd4a0bef1c311276657)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:48.850376 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75be62b7-e6ec-4038-ad4b-2d9cc3dfd0ee admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:49.024696 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1112/2222] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:38:48 2022] PUT /identity/v3/projects/75cd546fbdcb4d538a2284cca8c4637e/users/fa29d89f711341b3922810681eb564da/roles/4318f86ba9f74cd4a0bef1c311276657 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:38:49.026287 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ddef06-f18c-476c-baa4-ec0f78e1f59c admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:49.039450 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ddef06-f18c-476c-baa4-ec0f78e1f59c admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:49.200888 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c7472a-00e4-4ccb-abb8-0a32f5bcc77c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:49.201946 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c7472a-00e4-4ccb-abb8-0a32f5bcc77c admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:49.202767 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c7472a-00e4-4ccb-abb8-0a32f5bcc77c admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:49.203208 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c7472a-00e4-4ccb-abb8-0a32f5bcc77c admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:49.204425 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c7472a-00e4-4ccb-abb8-0a32f5bcc77c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:49.225918 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c7472a-00e4-4ccb-abb8-0a32f5bcc77c admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:49.263816 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1111/2223] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:38:49 2022] GET /identity/v3/roles => generated 1420 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:38:49.266962 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75be62b7-e6ec-4038-ad4b-2d9cc3dfd0ee admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:49.283421 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75be62b7-e6ec-4038-ad4b-2d9cc3dfd0ee admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:49.289432 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c891f379-84bb-4f36-a18f-c39300216e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:49.290374 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c891f379-84bb-4f36-a18f-c39300216e2d admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:49.290660 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c891f379-84bb-4f36-a18f-c39300216e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:49.290941 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c891f379-84bb-4f36-a18f-c39300216e2d admin admin] PATH_INFO: `/v3/projects/75cd546fbdcb4d538a2284cca8c4637e/users/fa29d89f711341b3922810681eb564da/roles/4318f86ba9f74cd4a0bef1c311276657` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:49.293147 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c891f379-84bb-4f36-a18f-c39300216e2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=75cd546fbdcb4d538a2284cca8c4637e, user_id=fa29d89f711341b3922810681eb564da, role_id=4318f86ba9f74cd4a0bef1c311276657)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:49.304758 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c891f379-84bb-4f36-a18f-c39300216e2d admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:49.312379 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1113/2224] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:38:49 2022] PUT /identity/v3/projects/75cd546fbdcb4d538a2284cca8c4637e/users/fa29d89f711341b3922810681eb564da/roles/4318f86ba9f74cd4a0bef1c311276657 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:38:49.326048 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b329b9fc-0007-44b3-b619-59f12e11bde7 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:49.326424 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b329b9fc-0007-44b3-b619-59f12e11bde7 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:49.326772 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b329b9fc-0007-44b3-b619-59f12e11bde7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:49.365505 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-b329b9fc-0007-44b3-b619-59f12e11bde7 None None] MFA Rules not processed for user `fa29d89f711341b3922810681eb564da`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:38:49.367122 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b329b9fc-0007-44b3-b619-59f12e11bde7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:49.489691 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1112/2225] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:38:49 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:38:49.505535 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c891f379-84bb-4f36-a18f-c39300216e2d admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:49.511258 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-c891f379-84bb-4f36-a18f-c39300216e2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:49.594691 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c891f379-84bb-4f36-a18f-c39300216e2d admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:49.599422 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0239f4e0-f1d9-4cdf-899a-55a6f999f0d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86c2df3d378490ca7ea497a73c19d40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:49.600584 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0239f4e0-f1d9-4cdf-899a-55a6f999f0d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:49.601098 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0239f4e0-f1d9-4cdf-899a-55a6f999f0d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:49.601580 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0239f4e0-f1d9-4cdf-899a-55a6f999f0d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:49.613014 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0239f4e0-f1d9-4cdf-899a-55a6f999f0d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:49.614488 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0239f4e0-f1d9-4cdf-899a-55a6f999f0d3 service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:49.626052 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1114/2226] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:38:49 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:38:49.653102 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:38:51.144461 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b329b9fc-0007-44b3-b619-59f12e11bde7 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:51.146173 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b329b9fc-0007-44b3-b619-59f12e11bde7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:51.159380 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b329b9fc-0007-44b3-b619-59f12e11bde7 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:51.162912 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4002d672-be72-4246-bf0f-55ce51409741 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86c2df3d378490ca7ea497a73c19d40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:51.163586 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4002d672-be72-4246-bf0f-55ce51409741 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:51.163868 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4002d672-be72-4246-bf0f-55ce51409741 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:51.164172 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4002d672-be72-4246-bf0f-55ce51409741 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:51.227024 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4002d672-be72-4246-bf0f-55ce51409741 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:51.228502 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4002d672-be72-4246-bf0f-55ce51409741 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:51.238797 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1113/2227] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:38:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:38:51.307945 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:38:52.447797 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:38:54.038751 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:38:55.707478 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:INSERT=5,keystone:SELECT=88,keystone:DELETE=7 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:38:56.081480 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:38:58.755418 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:38:59.035758 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:38:59.716353 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016215f6-16b1-4285-ad4b-a7369d5dbcf5 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:59.716567 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016215f6-16b1-4285-ad4b-a7369d5dbcf5 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:59.717222 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016215f6-16b1-4285-ad4b-a7369d5dbcf5 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:59.718735 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1115/2228] 10.222.0.33 () {54 vars in 921 bytes} [Fri Jul 15 19:38:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:38:59.729177 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c681e14-6fcb-45d1-807e-6b787bf76105 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:59.729473 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c681e14-6fcb-45d1-807e-6b787bf76105 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:59.729758 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c681e14-6fcb-45d1-807e-6b787bf76105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:59.758798 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-4c681e14-6fcb-45d1-807e-6b787bf76105 None None] MFA Rules not processed for user `52a0fce231ba4785a685ac9898d07503`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:38:59.760102 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-4c681e14-6fcb-45d1-807e-6b787bf76105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:59.835274 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1114/2229] 10.222.0.33 () {58 vars in 1018 bytes} [Fri Jul 15 19:38:59 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:38:59.852139 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016215f6-16b1-4285-ad4b-a7369d5dbcf5 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:38:59.853944 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-016215f6-16b1-4285-ad4b-a7369d5dbcf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:38:59.871148 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016215f6-16b1-4285-ad4b-a7369d5dbcf5 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:38:59.881139 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41583448-7d92-4738-a03a-021890c424df service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52a0fce231ba4785a685ac9898d07503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:38:59.882100 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41583448-7d92-4738-a03a-021890c424df service placement] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:38:59.882392 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41583448-7d92-4738-a03a-021890c424df service placement] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:38:59.882675 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41583448-7d92-4738-a03a-021890c424df service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:38:59.894184 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41583448-7d92-4738-a03a-021890c424df service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:38:59.895903 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41583448-7d92-4738-a03a-021890c424df service placement] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:38:59.909808 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1116/2230] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:38:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:39:00.031387 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c681e14-6fcb-45d1-807e-6b787bf76105 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:39:00.041734 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c681e14-6fcb-45d1-807e-6b787bf76105 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:39:00.045710 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877c772f-2084-487f-a0c9-22e5efe7c39c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86c2df3d378490ca7ea497a73c19d40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:39:00.046402 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877c772f-2084-487f-a0c9-22e5efe7c39c service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:00.046700 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877c772f-2084-487f-a0c9-22e5efe7c39c service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:00.046993 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877c772f-2084-487f-a0c9-22e5efe7c39c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:00.113050 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877c772f-2084-487f-a0c9-22e5efe7c39c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:39:00.114519 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877c772f-2084-487f-a0c9-22e5efe7c39c service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:39:00.123844 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1115/2231] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:39:00 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:39:00.301251 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:39:00.518201 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:39:00.691635 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41583448-7d92-4738-a03a-021890c424df service placement] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:39:00.693424 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-41583448-7d92-4738-a03a-021890c424df service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:39:00.761517 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41583448-7d92-4738-a03a-021890c424df service placement] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:39:00.766254 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380d0990-cce0-4a1e-99a4-a42b1a9ec90e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:39:00.767132 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380d0990-cce0-4a1e-99a4-a42b1a9ec90e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:00.767422 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380d0990-cce0-4a1e-99a4-a42b1a9ec90e admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:00.767700 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380d0990-cce0-4a1e-99a4-a42b1a9ec90e admin admin] PATH_INFO: `/v3/users/918bf5031e8e415ca5ef908a180855a2` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:00.768888 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380d0990-cce0-4a1e-99a4-a42b1a9ec90e admin admin] RBAC: Authorizing `identity:delete_user(user_id=918bf5031e8e415ca5ef908a180855a2)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:39:00.770866 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380d0990-cce0-4a1e-99a4-a42b1a9ec90e admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:39:01.017677 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-380d0990-cce0-4a1e-99a4-a42b1a9ec90e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '918bf5031e8e415ca5ef908a180855a2'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:01.029375 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-380d0990-cce0-4a1e-99a4-a42b1a9ec90e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '918bf5031e8e415ca5ef908a180855a2'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:01.132475 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-380d0990-cce0-4a1e-99a4-a42b1a9ec90e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '918bf5031e8e415ca5ef908a180855a2'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:01.132888 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-380d0990-cce0-4a1e-99a4-a42b1a9ec90e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '918bf5031e8e415ca5ef908a180855a2'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:01.133527 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-380d0990-cce0-4a1e-99a4-a42b1a9ec90e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '918bf5031e8e415ca5ef908a180855a2'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:01.146458 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1117/2232] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jul 15 19:39:00 2022] DELETE /identity/v3/users/918bf5031e8e415ca5ef908a180855a2 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:39:01.461157 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877c772f-2084-487f-a0c9-22e5efe7c39c service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:39:01.473826 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877c772f-2084-487f-a0c9-22e5efe7c39c service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:39:01.547475 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0b3b40-2c4b-497d-95d2-5eb202f802d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:39:01.548367 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b3b40-2c4b-497d-95d2-5eb202f802d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:01.548655 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b3b40-2c4b-497d-95d2-5eb202f802d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:01.548931 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b3b40-2c4b-497d-95d2-5eb202f802d3 admin admin] PATH_INFO: `/v3/projects/d7280ca621b24228b4c729aa57fa88b4` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:01.550179 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0b3b40-2c4b-497d-95d2-5eb202f802d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7280ca621b24228b4c729aa57fa88b4)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:39:01.552096 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0b3b40-2c4b-497d-95d2-5eb202f802d3 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:39:01.703417 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ee0b3b40-2c4b-497d-95d2-5eb202f802d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7280ca621b24228b4c729aa57fa88b4', 'request_id': 'req-ee0b3b40-2c4b-497d-95d2-5eb202f802d3'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:01.814988 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ee0b3b40-2c4b-497d-95d2-5eb202f802d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7280ca621b24228b4c729aa57fa88b4', 'request_id': 'req-ee0b3b40-2c4b-497d-95d2-5eb202f802d3'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:01.821077 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ee0b3b40-2c4b-497d-95d2-5eb202f802d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7280ca621b24228b4c729aa57fa88b4', 'request_id': 'req-ee0b3b40-2c4b-497d-95d2-5eb202f802d3'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:01.833200 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ee0b3b40-2c4b-497d-95d2-5eb202f802d3 admin admin] The token cache is being invalidate because project d7280ca621b24228b4c729aa57fa88b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:39:01.833494 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ee0b3b40-2c4b-497d-95d2-5eb202f802d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:01.835733 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ee0b3b40-2c4b-497d-95d2-5eb202f802d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:01.838582 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1116/2233] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jul 15 19:39:01 2022] DELETE /identity/v3/projects/d7280ca621b24228b4c729aa57fa88b4 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:39:01.839612 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380d0990-cce0-4a1e-99a4-a42b1a9ec90e admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:39:01.841242 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-380d0990-cce0-4a1e-99a4-a42b1a9ec90e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:39:01.855358 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380d0990-cce0-4a1e-99a4-a42b1a9ec90e admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:39:01.922444 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004a2694-3c66-44c8-9328-b8265cf63c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:39:01.923371 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004a2694-3c66-44c8-9328-b8265cf63c6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:01.923799 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004a2694-3c66-44c8-9328-b8265cf63c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:01.924078 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004a2694-3c66-44c8-9328-b8265cf63c6e admin admin] PATH_INFO: `/v3/users/979b97da7c0c40d3a5de6bd165fb6ea1` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:01.925408 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004a2694-3c66-44c8-9328-b8265cf63c6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=979b97da7c0c40d3a5de6bd165fb6ea1)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:39:01.927675 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004a2694-3c66-44c8-9328-b8265cf63c6e admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:39:02.121167 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-004a2694-3c66-44c8-9328-b8265cf63c6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '979b97da7c0c40d3a5de6bd165fb6ea1'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:02.133108 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-004a2694-3c66-44c8-9328-b8265cf63c6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '979b97da7c0c40d3a5de6bd165fb6ea1'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:02.249710 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-004a2694-3c66-44c8-9328-b8265cf63c6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '979b97da7c0c40d3a5de6bd165fb6ea1'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:02.250471 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-004a2694-3c66-44c8-9328-b8265cf63c6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '979b97da7c0c40d3a5de6bd165fb6ea1'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:02.251583 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-004a2694-3c66-44c8-9328-b8265cf63c6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '979b97da7c0c40d3a5de6bd165fb6ea1'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:02.264925 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1118/2234] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jul 15 19:39:01 2022] DELETE /identity/v3/users/979b97da7c0c40d3a5de6bd165fb6ea1 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:39:02.443803 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:39:02.584959 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0b3b40-2c4b-497d-95d2-5eb202f802d3 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:39:02.586464 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-ee0b3b40-2c4b-497d-95d2-5eb202f802d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:39:02.601895 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0b3b40-2c4b-497d-95d2-5eb202f802d3 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:39:02.665448 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df6786c-b9d4-426a-9ddd-4d5334bf2f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:39:02.666375 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df6786c-b9d4-426a-9ddd-4d5334bf2f37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:02.666904 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df6786c-b9d4-426a-9ddd-4d5334bf2f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:02.668283 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:39:02.668755 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df6786c-b9d4-426a-9ddd-4d5334bf2f37 admin admin] PATH_INFO: `/v3/projects/a44a47a5541743b09e08a5752d42abc1` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:02.670129 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df6786c-b9d4-426a-9ddd-4d5334bf2f37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a44a47a5541743b09e08a5752d42abc1)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:39:02.672071 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df6786c-b9d4-426a-9ddd-4d5334bf2f37 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:39:02.797880 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4df6786c-b9d4-426a-9ddd-4d5334bf2f37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a44a47a5541743b09e08a5752d42abc1', 'request_id': 'req-4df6786c-b9d4-426a-9ddd-4d5334bf2f37'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:02.906794 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4df6786c-b9d4-426a-9ddd-4d5334bf2f37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a44a47a5541743b09e08a5752d42abc1', 'request_id': 'req-4df6786c-b9d4-426a-9ddd-4d5334bf2f37'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:02.913475 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4df6786c-b9d4-426a-9ddd-4d5334bf2f37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a44a47a5541743b09e08a5752d42abc1', 'request_id': 'req-4df6786c-b9d4-426a-9ddd-4d5334bf2f37'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:02.926568 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4df6786c-b9d4-426a-9ddd-4d5334bf2f37 admin admin] The token cache is being invalidate because project a44a47a5541743b09e08a5752d42abc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:39:02.926856 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4df6786c-b9d4-426a-9ddd-4d5334bf2f37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:02.927607 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4df6786c-b9d4-426a-9ddd-4d5334bf2f37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:02.929190 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1117/2235] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jul 15 19:39:02 2022] DELETE /identity/v3/projects/a44a47a5541743b09e08a5752d42abc1 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:39:02.934128 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc18df-1baa-4fb8-b38f-c929ce46f9df None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:02.934433 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc18df-1baa-4fb8-b38f-c929ce46f9df None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:02.934712 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc18df-1baa-4fb8-b38f-c929ce46f9df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:02.953450 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-6edc18df-1baa-4fb8-b38f-c929ce46f9df None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:39:02.954722 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-6edc18df-1baa-4fb8-b38f-c929ce46f9df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:39:03.020084 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1119/2236] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:39:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:39:03.027831 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d640de7b-a80c-4384-a714-312d42e3b480 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:03.028124 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d640de7b-a80c-4384-a714-312d42e3b480 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:03.028428 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d640de7b-a80c-4384-a714-312d42e3b480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:03.048041 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-d640de7b-a80c-4384-a714-312d42e3b480 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:39:03.053174 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d640de7b-a80c-4384-a714-312d42e3b480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:39:03.058592 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1118/2237] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:39:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:39:03.062852 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edc18df-1baa-4fb8-b38f-c929ce46f9df None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:39:03.072622 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edc18df-1baa-4fb8-b38f-c929ce46f9df None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:39:03.077302 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e6437e-8774-478a-8c0f-b763b4e05ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:39:03.077989 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e6437e-8774-478a-8c0f-b763b4e05ee7 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:03.078275 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e6437e-8774-478a-8c0f-b763b4e05ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:03.078550 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e6437e-8774-478a-8c0f-b763b4e05ee7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:03.079303 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e6437e-8774-478a-8c0f-b763b4e05ee7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:39:03.079610 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e6437e-8774-478a-8c0f-b763b4e05ee7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:39:03.081203 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e6437e-8774-478a-8c0f-b763b4e05ee7 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:39:03.093540 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1120/2238] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jul 15 19:39:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:39:03.098581 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d640de7b-a80c-4384-a714-312d42e3b480 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:39:03.108050 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d640de7b-a80c-4384-a714-312d42e3b480 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:39:03.113667 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b7b9ed-93d3-460f-a88b-ab55b170f74a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:39:03.114377 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b7b9ed-93d3-460f-a88b-ab55b170f74a admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:03.114641 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b7b9ed-93d3-460f-a88b-ab55b170f74a admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:03.114918 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b7b9ed-93d3-460f-a88b-ab55b170f74a admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:03.116268 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b7b9ed-93d3-460f-a88b-ab55b170f74a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:39:03.118263 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b7b9ed-93d3-460f-a88b-ab55b170f74a admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:39:03.270221 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1119/2239] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jul 15 19:39:03 2022] POST /identity/v3/projects => generated 373 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:39:03.274363 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e6437e-8774-478a-8c0f-b763b4e05ee7 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:39:03.285099 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e6437e-8774-478a-8c0f-b763b4e05ee7 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:39:03.340692 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fa8947-880b-4c0c-93ca-bf34ad674312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:39:03.341368 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fa8947-880b-4c0c-93ca-bf34ad674312 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:03.341653 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fa8947-880b-4c0c-93ca-bf34ad674312 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:03.341996 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fa8947-880b-4c0c-93ca-bf34ad674312 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:03.342649 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fa8947-880b-4c0c-93ca-bf34ad674312 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:39:03.344555 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fa8947-880b-4c0c-93ca-bf34ad674312 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:39:03.450251 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1121/2240] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jul 15 19:39:03 2022] POST /identity/v3/users => generated 287 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:39:03.454529 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b7b9ed-93d3-460f-a88b-ab55b170f74a admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:39:03.465267 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b7b9ed-93d3-460f-a88b-ab55b170f74a admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:39:03.470775 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5e380a-7f38-4907-ad7f-2605bf43981f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:39:03.471507 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e380a-7f38-4907-ad7f-2605bf43981f admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:03.471889 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e380a-7f38-4907-ad7f-2605bf43981f admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:03.472291 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e380a-7f38-4907-ad7f-2605bf43981f admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:03.473089 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5e380a-7f38-4907-ad7f-2605bf43981f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:39:03.474945 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5e380a-7f38-4907-ad7f-2605bf43981f admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:39:03.488467 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1120/2241] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:39:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:39:03.492636 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fa8947-880b-4c0c-93ca-bf34ad674312 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:39:03.505671 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fa8947-880b-4c0c-93ca-bf34ad674312 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:39:03.510904 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40efad1-5ea6-4c04-b5a0-11524c58a29f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:39:03.511567 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40efad1-5ea6-4c04-b5a0-11524c58a29f admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:03.511962 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40efad1-5ea6-4c04-b5a0-11524c58a29f admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:03.512107 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40efad1-5ea6-4c04-b5a0-11524c58a29f admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:03.512842 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40efad1-5ea6-4c04-b5a0-11524c58a29f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:39:03.514581 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40efad1-5ea6-4c04-b5a0-11524c58a29f admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:39:03.525867 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1122/2242] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:39:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:39:03.529625 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5e380a-7f38-4907-ad7f-2605bf43981f admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:39:03.540072 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5e380a-7f38-4907-ad7f-2605bf43981f admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:39:03.546333 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3b00c5-9c21-4903-a76d-900aa604c421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:39:03.547407 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3b00c5-9c21-4903-a76d-900aa604c421 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:03.547705 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3b00c5-9c21-4903-a76d-900aa604c421 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:03.548494 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3b00c5-9c21-4903-a76d-900aa604c421 admin admin] PATH_INFO: `/v3/projects/c337285461424bc7894befc75e0a1348/users/5be9566db4cb4fbf96563257b3aaf96f/roles/28763c6e70094116b3a4b2895895fa58` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:03.564906 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3b00c5-9c21-4903-a76d-900aa604c421 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c337285461424bc7894befc75e0a1348, user_id=5be9566db4cb4fbf96563257b3aaf96f, role_id=28763c6e70094116b3a4b2895895fa58)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:39:03.569590 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3b00c5-9c21-4903-a76d-900aa604c421 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:39:03.652271 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1121/2243] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:39:03 2022] PUT /identity/v3/projects/c337285461424bc7894befc75e0a1348/users/5be9566db4cb4fbf96563257b3aaf96f/roles/28763c6e70094116b3a4b2895895fa58 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:39:03.663796 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aed3f9-6781-4e9a-b254-be8e241b865c None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:03.664088 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aed3f9-6781-4e9a-b254-be8e241b865c None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:03.664367 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aed3f9-6781-4e9a-b254-be8e241b865c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:03.692229 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-58aed3f9-6781-4e9a-b254-be8e241b865c None None] MFA Rules not processed for user `5be9566db4cb4fbf96563257b3aaf96f`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:39:03.693459 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-58aed3f9-6781-4e9a-b254-be8e241b865c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:39:03.758995 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1123/2244] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:39:03 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:39:03.769840 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3b00c5-9c21-4903-a76d-900aa604c421 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:39:03.771557 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-7b3b00c5-9c21-4903-a76d-900aa604c421 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:39:03.836215 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3b00c5-9c21-4903-a76d-900aa604c421 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:39:03.839940 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67438b26-a6e3-462a-b75b-62b3121dacc2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86c2df3d378490ca7ea497a73c19d40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:39:03.840588 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67438b26-a6e3-462a-b75b-62b3121dacc2 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:03.840872 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67438b26-a6e3-462a-b75b-62b3121dacc2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:03.841150 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67438b26-a6e3-462a-b75b-62b3121dacc2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:03.849679 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67438b26-a6e3-462a-b75b-62b3121dacc2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:39:03.851193 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67438b26-a6e3-462a-b75b-62b3121dacc2 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:39:03.860457 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1122/2245] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:39:03 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 19:39:04.609159 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:39:04.773779 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aed3f9-6781-4e9a-b254-be8e241b865c None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:39:04.775214 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-58aed3f9-6781-4e9a-b254-be8e241b865c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:39:04.789413 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:39:04.842307 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aed3f9-6781-4e9a-b254-be8e241b865c None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:39:04.846956 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1300f13-f58d-4382-b493-4241ffcc121f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:39:04.847886 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1300f13-f58d-4382-b493-4241ffcc121f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:04.848230 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1300f13-f58d-4382-b493-4241ffcc121f admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:04.848474 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1300f13-f58d-4382-b493-4241ffcc121f admin admin] PATH_INFO: `/v3/users/cb1a604bb9b24a5699d3bad4d369b654` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:04.849818 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1300f13-f58d-4382-b493-4241ffcc121f admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb1a604bb9b24a5699d3bad4d369b654)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:39:04.856833 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1300f13-f58d-4382-b493-4241ffcc121f admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:39:05.060821 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e1300f13-f58d-4382-b493-4241ffcc121f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb1a604bb9b24a5699d3bad4d369b654'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:05.073116 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e1300f13-f58d-4382-b493-4241ffcc121f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb1a604bb9b24a5699d3bad4d369b654'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:05.180883 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e1300f13-f58d-4382-b493-4241ffcc121f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb1a604bb9b24a5699d3bad4d369b654'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:05.181456 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e1300f13-f58d-4382-b493-4241ffcc121f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb1a604bb9b24a5699d3bad4d369b654'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:05.182100 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e1300f13-f58d-4382-b493-4241ffcc121f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb1a604bb9b24a5699d3bad4d369b654'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:05.195962 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1124/2246] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jul 15 19:39:04 2022] DELETE /identity/v3/users/cb1a604bb9b24a5699d3bad4d369b654 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:39:05.200563 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce46640-a414-4f63-abad-9f21b1aba9d4 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:05.200853 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce46640-a414-4f63-abad-9f21b1aba9d4 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:05.201133 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce46640-a414-4f63-abad-9f21b1aba9d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:05.219500 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-6ce46640-a414-4f63-abad-9f21b1aba9d4 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:39:05.220718 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-6ce46640-a414-4f63-abad-9f21b1aba9d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:39:05.280092 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1123/2247] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:39:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:39:05.290464 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1300f13-f58d-4382-b493-4241ffcc121f admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:39:05.291863 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-e1300f13-f58d-4382-b493-4241ffcc121f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:39:05.354634 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1300f13-f58d-4382-b493-4241ffcc121f admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:39:05.358539 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecd4137-52ce-4801-abfc-db2c408f8b5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:39:05.359311 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecd4137-52ce-4801-abfc-db2c408f8b5b service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:05.359746 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecd4137-52ce-4801-abfc-db2c408f8b5b service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:05.360098 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecd4137-52ce-4801-abfc-db2c408f8b5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:05.369738 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecd4137-52ce-4801-abfc-db2c408f8b5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:39:05.371200 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecd4137-52ce-4801-abfc-db2c408f8b5b service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:39:05.381046 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1125/2248] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:39:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:39:05.936362 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce46640-a414-4f63-abad-9f21b1aba9d4 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:39:05.937917 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-6ce46640-a414-4f63-abad-9f21b1aba9d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:39:05.951829 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce46640-a414-4f63-abad-9f21b1aba9d4 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:39:05.957737 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0613cedb-7e5f-43c5-a815-13e33e98363e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:39:05.959468 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0613cedb-7e5f-43c5-a815-13e33e98363e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:05.960104 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0613cedb-7e5f-43c5-a815-13e33e98363e admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:05.960395 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0613cedb-7e5f-43c5-a815-13e33e98363e admin admin] PATH_INFO: `/v3/projects/3c5efd83a79f410b8ecd163af9e19cff` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:05.961835 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0613cedb-7e5f-43c5-a815-13e33e98363e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c5efd83a79f410b8ecd163af9e19cff)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:39:05.963801 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0613cedb-7e5f-43c5-a815-13e33e98363e admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:39:06.083382 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-0613cedb-7e5f-43c5-a815-13e33e98363e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c5efd83a79f410b8ecd163af9e19cff', 'request_id': 'req-0613cedb-7e5f-43c5-a815-13e33e98363e'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:06.132497 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:39:06.174498 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-0613cedb-7e5f-43c5-a815-13e33e98363e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c5efd83a79f410b8ecd163af9e19cff', 'request_id': 'req-0613cedb-7e5f-43c5-a815-13e33e98363e'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:06.180709 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-0613cedb-7e5f-43c5-a815-13e33e98363e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c5efd83a79f410b8ecd163af9e19cff', 'request_id': 'req-0613cedb-7e5f-43c5-a815-13e33e98363e'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:06.193187 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-0613cedb-7e5f-43c5-a815-13e33e98363e admin admin] The token cache is being invalidate because project 3c5efd83a79f410b8ecd163af9e19cff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:39:06.193470 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-0613cedb-7e5f-43c5-a815-13e33e98363e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:06.194136 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-0613cedb-7e5f-43c5-a815-13e33e98363e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:06.195725 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1124/2249] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jul 15 19:39:05 2022] DELETE /identity/v3/projects/3c5efd83a79f410b8ecd163af9e19cff => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:39:06.199883 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecd4137-52ce-4801-abfc-db2c408f8b5b service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:39:06.210320 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecd4137-52ce-4801-abfc-db2c408f8b5b service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:39:06.275128 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8409ab8a-ab80-4c68-babd-abe8e10a80b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:39:06.276022 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8409ab8a-ab80-4c68-babd-abe8e10a80b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:06.276308 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8409ab8a-ab80-4c68-babd-abe8e10a80b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:06.276585 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8409ab8a-ab80-4c68-babd-abe8e10a80b8 admin admin] PATH_INFO: `/v3/users/da37136a00e34742ab85d94315938fbc` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:06.277885 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8409ab8a-ab80-4c68-babd-abe8e10a80b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da37136a00e34742ab85d94315938fbc)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:39:06.279865 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8409ab8a-ab80-4c68-babd-abe8e10a80b8 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:39:06.548653 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-8409ab8a-ab80-4c68-babd-abe8e10a80b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da37136a00e34742ab85d94315938fbc'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:06.560831 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-8409ab8a-ab80-4c68-babd-abe8e10a80b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da37136a00e34742ab85d94315938fbc'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:06.660679 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:39:06.680708 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-8409ab8a-ab80-4c68-babd-abe8e10a80b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da37136a00e34742ab85d94315938fbc'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:06.681346 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-8409ab8a-ab80-4c68-babd-abe8e10a80b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da37136a00e34742ab85d94315938fbc'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:06.682007 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-8409ab8a-ab80-4c68-babd-abe8e10a80b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da37136a00e34742ab85d94315938fbc'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:06.696230 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1126/2250] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jul 15 19:39:06 2022] DELETE /identity/v3/users/da37136a00e34742ab85d94315938fbc => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:39:07.136681 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0613cedb-7e5f-43c5-a815-13e33e98363e admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:39:07.138556 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0613cedb-7e5f-43c5-a815-13e33e98363e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:39:07.153665 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0613cedb-7e5f-43c5-a815-13e33e98363e admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:39:07.216217 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f251145-687c-4703-8eb9-f74d9df6f0ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:39:07.217097 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f251145-687c-4703-8eb9-f74d9df6f0ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:07.217398 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f251145-687c-4703-8eb9-f74d9df6f0ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:07.217684 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f251145-687c-4703-8eb9-f74d9df6f0ed admin admin] PATH_INFO: `/v3/projects/723bd820fa9e40159e90c0650a7c4516` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:07.218897 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f251145-687c-4703-8eb9-f74d9df6f0ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=723bd820fa9e40159e90c0650a7c4516)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:39:07.220839 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f251145-687c-4703-8eb9-f74d9df6f0ed admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:39:07.449437 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-0f251145-687c-4703-8eb9-f74d9df6f0ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '723bd820fa9e40159e90c0650a7c4516', 'request_id': 'req-0f251145-687c-4703-8eb9-f74d9df6f0ed'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:07.592157 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-0f251145-687c-4703-8eb9-f74d9df6f0ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '723bd820fa9e40159e90c0650a7c4516', 'request_id': 'req-0f251145-687c-4703-8eb9-f74d9df6f0ed'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:07.598706 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-0f251145-687c-4703-8eb9-f74d9df6f0ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '723bd820fa9e40159e90c0650a7c4516', 'request_id': 'req-0f251145-687c-4703-8eb9-f74d9df6f0ed'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:07.609650 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-0f251145-687c-4703-8eb9-f74d9df6f0ed admin admin] The token cache is being invalidate because project 723bd820fa9e40159e90c0650a7c4516 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:39:07.609966 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-0f251145-687c-4703-8eb9-f74d9df6f0ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:07.611144 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-0f251145-687c-4703-8eb9-f74d9df6f0ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:07.612703 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1125/2251] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jul 15 19:39:07 2022] DELETE /identity/v3/projects/723bd820fa9e40159e90c0650a7c4516 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:39:07.616765 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8409ab8a-ab80-4c68-babd-abe8e10a80b8 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:39:07.618226 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-8409ab8a-ab80-4c68-babd-abe8e10a80b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:39:07.631044 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8409ab8a-ab80-4c68-babd-abe8e10a80b8 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:39:07.646866 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:39:07.688466 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55128cfb-0f83-4b19-a984-9010147764d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:39:07.689350 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55128cfb-0f83-4b19-a984-9010147764d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:07.689635 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55128cfb-0f83-4b19-a984-9010147764d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:07.689968 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55128cfb-0f83-4b19-a984-9010147764d9 admin admin] PATH_INFO: `/v3/users/fa29d89f711341b3922810681eb564da` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:07.691166 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55128cfb-0f83-4b19-a984-9010147764d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa29d89f711341b3922810681eb564da)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:39:07.693094 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55128cfb-0f83-4b19-a984-9010147764d9 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:39:08.014064 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-55128cfb-0f83-4b19-a984-9010147764d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa29d89f711341b3922810681eb564da'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:08.029931 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-55128cfb-0f83-4b19-a984-9010147764d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa29d89f711341b3922810681eb564da'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:08.106291 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-55128cfb-0f83-4b19-a984-9010147764d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa29d89f711341b3922810681eb564da'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:08.107040 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-55128cfb-0f83-4b19-a984-9010147764d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa29d89f711341b3922810681eb564da'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:08.108083 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-55128cfb-0f83-4b19-a984-9010147764d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa29d89f711341b3922810681eb564da'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:08.123892 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1127/2252] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jul 15 19:39:07 2022] DELETE /identity/v3/users/fa29d89f711341b3922810681eb564da => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:39:08.201067 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:39:08.355569 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:39:08.624660 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f251145-687c-4703-8eb9-f74d9df6f0ed admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:39:08.626707 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0f251145-687c-4703-8eb9-f74d9df6f0ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:39:08.640493 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f251145-687c-4703-8eb9-f74d9df6f0ed admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:39:08.697713 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e539627-ae5e-419f-8567-d8c29a28f542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:39:08.698626 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e539627-ae5e-419f-8567-d8c29a28f542 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:08.698911 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e539627-ae5e-419f-8567-d8c29a28f542 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:08.699191 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e539627-ae5e-419f-8567-d8c29a28f542 admin admin] PATH_INFO: `/v3/projects/75cd546fbdcb4d538a2284cca8c4637e` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:08.700379 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e539627-ae5e-419f-8567-d8c29a28f542 admin admin] RBAC: Authorizing `identity:delete_project(project_id=75cd546fbdcb4d538a2284cca8c4637e)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:39:08.702334 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e539627-ae5e-419f-8567-d8c29a28f542 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:39:08.824674 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4e539627-ae5e-419f-8567-d8c29a28f542 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75cd546fbdcb4d538a2284cca8c4637e', 'request_id': 'req-4e539627-ae5e-419f-8567-d8c29a28f542'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:08.943512 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4e539627-ae5e-419f-8567-d8c29a28f542 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75cd546fbdcb4d538a2284cca8c4637e', 'request_id': 'req-4e539627-ae5e-419f-8567-d8c29a28f542'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:08.949991 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4e539627-ae5e-419f-8567-d8c29a28f542 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75cd546fbdcb4d538a2284cca8c4637e', 'request_id': 'req-4e539627-ae5e-419f-8567-d8c29a28f542'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:08.962911 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4e539627-ae5e-419f-8567-d8c29a28f542 admin admin] The token cache is being invalidate because project 75cd546fbdcb4d538a2284cca8c4637e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:39:08.963201 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4e539627-ae5e-419f-8567-d8c29a28f542 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:08.963852 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4e539627-ae5e-419f-8567-d8c29a28f542 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:08.965566 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1126/2253] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jul 15 19:39:08 2022] DELETE /identity/v3/projects/75cd546fbdcb4d538a2284cca8c4637e => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:39:08.972015 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3163bad2-da1e-4a1a-9dd2-5e1a00b5fa0a None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:08.972481 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3163bad2-da1e-4a1a-9dd2-5e1a00b5fa0a None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:08.972576 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3163bad2-da1e-4a1a-9dd2-5e1a00b5fa0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:08.991145 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-3163bad2-da1e-4a1a-9dd2-5e1a00b5fa0a None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:39:08.992371 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-3163bad2-da1e-4a1a-9dd2-5e1a00b5fa0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:39:09.051732 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1128/2254] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:39:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:39:09.060289 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d0e6f3-8754-482d-be82-31dabef46a49 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:09.060577 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d0e6f3-8754-482d-be82-31dabef46a49 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:09.060932 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d0e6f3-8754-482d-be82-31dabef46a49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:09.080872 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-49d0e6f3-8754-482d-be82-31dabef46a49 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:39:09.082106 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-49d0e6f3-8754-482d-be82-31dabef46a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:39:09.090228 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1127/2255] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:39:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:39:09.094419 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3163bad2-da1e-4a1a-9dd2-5e1a00b5fa0a None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:39:09.104181 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3163bad2-da1e-4a1a-9dd2-5e1a00b5fa0a None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:39:09.108831 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3380f68-e665-472f-b1d4-8a878f7d8f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:39:09.109484 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3380f68-e665-472f-b1d4-8a878f7d8f91 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:09.110599 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3380f68-e665-472f-b1d4-8a878f7d8f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:09.110599 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3380f68-e665-472f-b1d4-8a878f7d8f91 admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:09.110817 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3380f68-e665-472f-b1d4-8a878f7d8f91 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:39:09.111081 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3380f68-e665-472f-b1d4-8a878f7d8f91 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:39:09.112690 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3380f68-e665-472f-b1d4-8a878f7d8f91 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:39:09.124186 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1129/2256] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jul 15 19:39:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:39:09.128864 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d0e6f3-8754-482d-be82-31dabef46a49 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:39:09.138369 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d0e6f3-8754-482d-be82-31dabef46a49 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:39:09.143051 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f9616b-14c2-42f4-818b-8d554a312ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:39:09.143698 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f9616b-14c2-42f4-818b-8d554a312ebe admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:09.145186 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f9616b-14c2-42f4-818b-8d554a312ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:09.145186 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f9616b-14c2-42f4-818b-8d554a312ebe admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:09.145186 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f9616b-14c2-42f4-818b-8d554a312ebe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:39:09.146845 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f9616b-14c2-42f4-818b-8d554a312ebe admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:39:09.284828 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1128/2257] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jul 15 19:39:09 2022] POST /identity/v3/projects => generated 401 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:39:09.289187 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3380f68-e665-472f-b1d4-8a878f7d8f91 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:39:09.300165 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3380f68-e665-472f-b1d4-8a878f7d8f91 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:39:09.356143 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf55730-7f73-415b-8def-c98a10c35449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:39:09.356945 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf55730-7f73-415b-8def-c98a10c35449 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:09.357261 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf55730-7f73-415b-8def-c98a10c35449 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:09.357543 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf55730-7f73-415b-8def-c98a10c35449 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:09.358307 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf55730-7f73-415b-8def-c98a10c35449 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:39:09.360758 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf55730-7f73-415b-8def-c98a10c35449 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:39:09.438947 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f9616b-14c2-42f4-818b-8d554a312ebe admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:39:09.441059 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-70f9616b-14c2-42f4-818b-8d554a312ebe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:39:09.455348 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f9616b-14c2-42f4-818b-8d554a312ebe admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:39:09.460135 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d71218-13df-4242-a6d8-95931d31cb2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:39:09.465645 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d71218-13df-4242-a6d8-95931d31cb2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:09.465645 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d71218-13df-4242-a6d8-95931d31cb2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:09.465645 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d71218-13df-4242-a6d8-95931d31cb2b admin admin] PATH_INFO: `/v3/users/5be9566db4cb4fbf96563257b3aaf96f` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:09.465645 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d71218-13df-4242-a6d8-95931d31cb2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5be9566db4cb4fbf96563257b3aaf96f)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:39:09.465645 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d71218-13df-4242-a6d8-95931d31cb2b admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:39:09.466422 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1130/2258] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jul 15 19:39:09 2022] POST /identity/v3/users => generated 301 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:39:09.471176 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf55730-7f73-415b-8def-c98a10c35449 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:39:09.487577 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf55730-7f73-415b-8def-c98a10c35449 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:39:09.492359 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adfaccf-dcb9-4033-99c2-f442484801de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:39:09.493221 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adfaccf-dcb9-4033-99c2-f442484801de admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:09.493506 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adfaccf-dcb9-4033-99c2-f442484801de admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:09.493804 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adfaccf-dcb9-4033-99c2-f442484801de admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:09.494551 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adfaccf-dcb9-4033-99c2-f442484801de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:39:09.496139 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adfaccf-dcb9-4033-99c2-f442484801de admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:39:09.507349 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1131/2259] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:39:09 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 19:39:09.511296 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adfaccf-dcb9-4033-99c2-f442484801de admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:39:09.520936 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adfaccf-dcb9-4033-99c2-f442484801de admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:39:09.525640 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10966d16-4b72-482a-aefb-908d899f2851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:39:09.526316 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10966d16-4b72-482a-aefb-908d899f2851 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:09.526603 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10966d16-4b72-482a-aefb-908d899f2851 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:09.526883 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10966d16-4b72-482a-aefb-908d899f2851 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:09.527601 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10966d16-4b72-482a-aefb-908d899f2851 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:39:09.529182 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10966d16-4b72-482a-aefb-908d899f2851 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:39:09.539878 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1132/2260] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:39:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 19:39:09.543611 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10966d16-4b72-482a-aefb-908d899f2851 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:39:09.554062 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10966d16-4b72-482a-aefb-908d899f2851 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:39:09.558800 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e952078a-497b-4862-be62-a86fb2bc80de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:39:09.559646 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e952078a-497b-4862-be62-a86fb2bc80de admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:09.559933 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e952078a-497b-4862-be62-a86fb2bc80de admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:09.560207 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e952078a-497b-4862-be62-a86fb2bc80de admin admin] PATH_INFO: `/v3/projects/6fc539080bff4995a1d6b24ef2abf0aa/users/49fac8536563457ba7e48446b2d62708/roles/28763c6e70094116b3a4b2895895fa58` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:09.575324 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e952078a-497b-4862-be62-a86fb2bc80de admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fc539080bff4995a1d6b24ef2abf0aa, user_id=49fac8536563457ba7e48446b2d62708, role_id=28763c6e70094116b3a4b2895895fa58)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:39:09.577973 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e952078a-497b-4862-be62-a86fb2bc80de admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:39:09.749586 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1133/2261] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:39:09 2022] PUT /identity/v3/projects/6fc539080bff4995a1d6b24ef2abf0aa/users/49fac8536563457ba7e48446b2d62708/roles/28763c6e70094116b3a4b2895895fa58 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:39:09.762822 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fd5ec7-6978-4613-947c-32618351024e None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:09.763714 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fd5ec7-6978-4613-947c-32618351024e None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:09.763714 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fd5ec7-6978-4613-947c-32618351024e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:09.777499 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-76d71218-13df-4242-a6d8-95931d31cb2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5be9566db4cb4fbf96563257b3aaf96f'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:09.786185 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-76d71218-13df-4242-a6d8-95931d31cb2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5be9566db4cb4fbf96563257b3aaf96f'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:09.794092 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-a7fd5ec7-6978-4613-947c-32618351024e None None] MFA Rules not processed for user `49fac8536563457ba7e48446b2d62708`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:39:09.795410 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-a7fd5ec7-6978-4613-947c-32618351024e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:39:09.809434 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:39:09.871039 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1134/2262] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:39:09 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:39:09.878796 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-76d71218-13df-4242-a6d8-95931d31cb2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5be9566db4cb4fbf96563257b3aaf96f'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:09.878796 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-76d71218-13df-4242-a6d8-95931d31cb2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5be9566db4cb4fbf96563257b3aaf96f'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:09.883035 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fd5ec7-6978-4613-947c-32618351024e None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:39:09.887837 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-76d71218-13df-4242-a6d8-95931d31cb2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5be9566db4cb4fbf96563257b3aaf96f'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:09.898757 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1129/2263] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jul 15 19:39:09 2022] DELETE /identity/v3/users/5be9566db4cb4fbf96563257b3aaf96f => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:39:09.903508 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce95e82-3a48-47be-841e-d20ca1f0ffba None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:09.903904 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce95e82-3a48-47be-841e-d20ca1f0ffba None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:09.904252 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce95e82-3a48-47be-841e-d20ca1f0ffba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:09.935270 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-8ce95e82-3a48-47be-841e-d20ca1f0ffba None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:39:09.936603 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-8ce95e82-3a48-47be-841e-d20ca1f0ffba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:39:09.975939 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fd5ec7-6978-4613-947c-32618351024e None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:39:10.008024 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1130/2264] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:39:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:39:10.017987 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce95e82-3a48-47be-841e-d20ca1f0ffba None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:39:10.019403 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-8ce95e82-3a48-47be-841e-d20ca1f0ffba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:39:10.035578 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ba1c53-dbe9-4049-8a0c-080fa03b90d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:39:10.036554 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ba1c53-dbe9-4049-8a0c-080fa03b90d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:10.036844 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ba1c53-dbe9-4049-8a0c-080fa03b90d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:10.037122 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ba1c53-dbe9-4049-8a0c-080fa03b90d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:10.038419 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ba1c53-dbe9-4049-8a0c-080fa03b90d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:39:10.040629 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ba1c53-dbe9-4049-8a0c-080fa03b90d1 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:39:10.086064 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce95e82-3a48-47be-841e-d20ca1f0ffba None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:39:10.089727 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae6cae1-ae3e-4356-b3d9-ea9ced53a89a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:39:10.090409 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6cae1-ae3e-4356-b3d9-ea9ced53a89a service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:10.090689 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6cae1-ae3e-4356-b3d9-ea9ced53a89a service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:10.090964 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6cae1-ae3e-4356-b3d9-ea9ced53a89a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:10.100005 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae6cae1-ae3e-4356-b3d9-ea9ced53a89a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:39:10.101434 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae6cae1-ae3e-4356-b3d9-ea9ced53a89a service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:39:10.111394 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1131/2265] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:39:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:39:10.296611 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1135/2266] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jul 15 19:39:09 2022] POST /identity/v3/projects => generated 401 bytes in 414 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:39:10.300941 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae6cae1-ae3e-4356-b3d9-ea9ced53a89a service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:39:10.302393 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-4ae6cae1-ae3e-4356-b3d9-ea9ced53a89a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:39:10.315753 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae6cae1-ae3e-4356-b3d9-ea9ced53a89a service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:39:10.370325 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85184fe2-0317-4a89-b840-f464e0cf7055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:39:10.370994 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85184fe2-0317-4a89-b840-f464e0cf7055 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:10.371280 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85184fe2-0317-4a89-b840-f464e0cf7055 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:10.371558 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85184fe2-0317-4a89-b840-f464e0cf7055 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:10.372237 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85184fe2-0317-4a89-b840-f464e0cf7055 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:39:10.374305 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85184fe2-0317-4a89-b840-f464e0cf7055 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:39:10.610981 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1132/2267] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jul 15 19:39:10 2022] POST /identity/v3/users => generated 307 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:39:10.611680 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ba1c53-dbe9-4049-8a0c-080fa03b90d1 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:39:10.611680 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ba1c53-dbe9-4049-8a0c-080fa03b90d1 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:39:10.615496 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d97883c-cbc0-456b-8984-9f9a4d21d94e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:39:10.616259 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d97883c-cbc0-456b-8984-9f9a4d21d94e admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:10.616539 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d97883c-cbc0-456b-8984-9f9a4d21d94e admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:10.616838 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d97883c-cbc0-456b-8984-9f9a4d21d94e admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:10.617565 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d97883c-cbc0-456b-8984-9f9a4d21d94e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:39:10.619220 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d97883c-cbc0-456b-8984-9f9a4d21d94e admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:39:10.629722 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1136/2268] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:39:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:39:10.634147 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85184fe2-0317-4a89-b840-f464e0cf7055 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:39:10.644219 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85184fe2-0317-4a89-b840-f464e0cf7055 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:39:10.648754 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942b58c1-1b43-4a2b-847e-7f16044a30e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:39:10.649626 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942b58c1-1b43-4a2b-847e-7f16044a30e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:10.649945 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942b58c1-1b43-4a2b-847e-7f16044a30e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:10.650225 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942b58c1-1b43-4a2b-847e-7f16044a30e5 admin admin] PATH_INFO: `/v3/projects/ed93f6a2878240539aaae22e65014111/users/b26293a1e8624ae591356ac3c02c47a4/roles/4318f86ba9f74cd4a0bef1c311276657` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:10.661061 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:39:10.667202 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942b58c1-1b43-4a2b-847e-7f16044a30e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed93f6a2878240539aaae22e65014111, user_id=b26293a1e8624ae591356ac3c02c47a4, role_id=4318f86ba9f74cd4a0bef1c311276657)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:39:10.668379 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d97883c-cbc0-456b-8984-9f9a4d21d94e admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:39:10.669900 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-0d97883c-cbc0-456b-8984-9f9a4d21d94e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:39:10.670410 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942b58c1-1b43-4a2b-847e-7f16044a30e5 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:39:10.689930 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d97883c-cbc0-456b-8984-9f9a4d21d94e admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:39:10.694979 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e6e8b6-ba0a-482c-98ac-353efc7e7248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:39:10.697101 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e6e8b6-ba0a-482c-98ac-353efc7e7248 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:10.697101 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e6e8b6-ba0a-482c-98ac-353efc7e7248 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:10.697101 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e6e8b6-ba0a-482c-98ac-353efc7e7248 admin admin] PATH_INFO: `/v3/projects/c337285461424bc7894befc75e0a1348` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:10.697971 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e6e8b6-ba0a-482c-98ac-353efc7e7248 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c337285461424bc7894befc75e0a1348)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:39:10.699723 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e6e8b6-ba0a-482c-98ac-353efc7e7248 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:39:10.721352 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:INSERT=3,keystone:SELECT=92,keystone:DELETE=5 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:39:10.758064 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1133/2269] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:39:10 2022] PUT /identity/v3/projects/ed93f6a2878240539aaae22e65014111/users/b26293a1e8624ae591356ac3c02c47a4/roles/4318f86ba9f74cd4a0bef1c311276657 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:39:10.762088 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942b58c1-1b43-4a2b-847e-7f16044a30e5 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:39:10.825494 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942b58c1-1b43-4a2b-847e-7f16044a30e5 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:39:10.826270 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-81e6e8b6-ba0a-482c-98ac-353efc7e7248 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c337285461424bc7894befc75e0a1348', 'request_id': 'req-81e6e8b6-ba0a-482c-98ac-353efc7e7248'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:10.881421 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65563ccf-372c-4b64-91f1-568a38ce04c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:39:10.882232 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65563ccf-372c-4b64-91f1-568a38ce04c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:10.882511 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65563ccf-372c-4b64-91f1-568a38ce04c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:10.882782 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65563ccf-372c-4b64-91f1-568a38ce04c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:10.883551 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65563ccf-372c-4b64-91f1-568a38ce04c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:39:10.885143 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65563ccf-372c-4b64-91f1-568a38ce04c8 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:39:10.895094 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1134/2270] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:39:10 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 19:39:10.899091 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65563ccf-372c-4b64-91f1-568a38ce04c8 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:39:10.909113 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65563ccf-372c-4b64-91f1-568a38ce04c8 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:39:10.914918 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfea4d75-b745-4897-9ab1-1d394704131a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:39:10.914918 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea4d75-b745-4897-9ab1-1d394704131a admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:10.914918 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea4d75-b745-4897-9ab1-1d394704131a admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:10.915225 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea4d75-b745-4897-9ab1-1d394704131a admin admin] PATH_INFO: `/v3/projects/ed93f6a2878240539aaae22e65014111/users/b26293a1e8624ae591356ac3c02c47a4/roles/4318f86ba9f74cd4a0bef1c311276657` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:10.917032 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfea4d75-b745-4897-9ab1-1d394704131a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed93f6a2878240539aaae22e65014111, user_id=b26293a1e8624ae591356ac3c02c47a4, role_id=4318f86ba9f74cd4a0bef1c311276657)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:39:10.919674 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfea4d75-b745-4897-9ab1-1d394704131a admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:39:10.929497 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1135/2271] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:39:10 2022] PUT /identity/v3/projects/ed93f6a2878240539aaae22e65014111/users/b26293a1e8624ae591356ac3c02c47a4/roles/4318f86ba9f74cd4a0bef1c311276657 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:39:10.939385 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2521c93-c63a-4a7d-b897-632d8d2daefc None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:10.939687 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2521c93-c63a-4a7d-b897-632d8d2daefc None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:10.939958 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2521c93-c63a-4a7d-b897-632d8d2daefc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:10.967784 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-e2521c93-c63a-4a7d-b897-632d8d2daefc None None] MFA Rules not processed for user `b26293a1e8624ae591356ac3c02c47a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:39:10.969204 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e2521c93-c63a-4a7d-b897-632d8d2daefc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:39:10.983662 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-81e6e8b6-ba0a-482c-98ac-353efc7e7248 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c337285461424bc7894befc75e0a1348', 'request_id': 'req-81e6e8b6-ba0a-482c-98ac-353efc7e7248'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:10.991627 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-81e6e8b6-ba0a-482c-98ac-353efc7e7248 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c337285461424bc7894befc75e0a1348', 'request_id': 'req-81e6e8b6-ba0a-482c-98ac-353efc7e7248'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:11.006384 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-81e6e8b6-ba0a-482c-98ac-353efc7e7248 admin admin] The token cache is being invalidate because project c337285461424bc7894befc75e0a1348 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:39:11.006384 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-81e6e8b6-ba0a-482c-98ac-353efc7e7248 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:11.006384 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-81e6e8b6-ba0a-482c-98ac-353efc7e7248 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:11.014889 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1137/2272] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jul 15 19:39:10 2022] DELETE /identity/v3/projects/c337285461424bc7894befc75e0a1348 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:39:11.020054 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d720e686-dac0-4018-b50c-70f8d04c19ac None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:11.020347 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d720e686-dac0-4018-b50c-70f8d04c19ac None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:11.020629 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d720e686-dac0-4018-b50c-70f8d04c19ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:11.040544 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-d720e686-dac0-4018-b50c-70f8d04c19ac None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:39:11.041927 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-d720e686-dac0-4018-b50c-70f8d04c19ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:39:11.042670 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1136/2273] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:39:10 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:39:11.054451 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2521c93-c63a-4a7d-b897-632d8d2daefc None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:39:11.055833 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e2521c93-c63a-4a7d-b897-632d8d2daefc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:39:11.104155 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1138/2274] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:39:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:39:11.112323 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6852f36-de53-4b7b-81ed-f1e5396e4a0c None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:11.112610 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6852f36-de53-4b7b-81ed-f1e5396e4a0c None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:11.112887 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6852f36-de53-4b7b-81ed-f1e5396e4a0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:11.115267 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2521c93-c63a-4a7d-b897-632d8d2daefc None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:39:11.119089 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d628e9-fcce-4b2e-b57d-98c1ed2694ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86c2df3d378490ca7ea497a73c19d40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:39:11.119823 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d628e9-fcce-4b2e-b57d-98c1ed2694ce service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:11.120193 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d628e9-fcce-4b2e-b57d-98c1ed2694ce service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:11.120494 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d628e9-fcce-4b2e-b57d-98c1ed2694ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:11.131274 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d628e9-fcce-4b2e-b57d-98c1ed2694ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:39:11.133293 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d628e9-fcce-4b2e-b57d-98c1ed2694ce service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:39:11.134168 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-d6852f36-de53-4b7b-81ed-f1e5396e4a0c None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:39:11.135683 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-d6852f36-de53-4b7b-81ed-f1e5396e4a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:39:11.144132 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1137/2275] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:39:11 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 19:39:11.144424 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1139/2276] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:39:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:39:11.148465 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6852f36-de53-4b7b-81ed-f1e5396e4a0c None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:39:11.157911 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6852f36-de53-4b7b-81ed-f1e5396e4a0c None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:39:11.163096 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c91012-2bef-4d48-95a5-45d404f2f977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:39:11.163817 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c91012-2bef-4d48-95a5-45d404f2f977 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:11.164101 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c91012-2bef-4d48-95a5-45d404f2f977 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:11.164381 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c91012-2bef-4d48-95a5-45d404f2f977 admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:11.165119 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c91012-2bef-4d48-95a5-45d404f2f977 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:39:11.165425 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c91012-2bef-4d48-95a5-45d404f2f977 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:39:11.167109 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c91012-2bef-4d48-95a5-45d404f2f977 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:39:11.178981 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1140/2277] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jul 15 19:39:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:39:11.183460 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d628e9-fcce-4b2e-b57d-98c1ed2694ce service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:39:11.194384 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d628e9-fcce-4b2e-b57d-98c1ed2694ce service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:39:11.199031 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affa22d9-6103-4db0-b66b-70a8d56e4f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:39:11.199679 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affa22d9-6103-4db0-b66b-70a8d56e4f16 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:11.199964 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affa22d9-6103-4db0-b66b-70a8d56e4f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:11.200242 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affa22d9-6103-4db0-b66b-70a8d56e4f16 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:11.200944 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affa22d9-6103-4db0-b66b-70a8d56e4f16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:39:11.202913 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affa22d9-6103-4db0-b66b-70a8d56e4f16 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:39:11.338228 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1138/2278] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jul 15 19:39:11 2022] POST /identity/v3/projects => generated 373 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:39:11.339299 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c91012-2bef-4d48-95a5-45d404f2f977 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:39:11.349039 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c91012-2bef-4d48-95a5-45d404f2f977 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:39:11.404825 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a490196c-137c-433f-95f3-0767dd821bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:39:11.405415 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a490196c-137c-433f-95f3-0767dd821bb6 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:11.405700 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a490196c-137c-433f-95f3-0767dd821bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:11.406009 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a490196c-137c-433f-95f3-0767dd821bb6 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:11.406728 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a490196c-137c-433f-95f3-0767dd821bb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:39:11.408645 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a490196c-137c-433f-95f3-0767dd821bb6 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:39:11.463430 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affa22d9-6103-4db0-b66b-70a8d56e4f16 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:39:11.525261 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affa22d9-6103-4db0-b66b-70a8d56e4f16 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:39:11.529205 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ffcc78-9b67-46d1-a092-3ad6c2c23178 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86c2df3d378490ca7ea497a73c19d40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:39:11.529911 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ffcc78-9b67-46d1-a092-3ad6c2c23178 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:11.530196 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ffcc78-9b67-46d1-a092-3ad6c2c23178 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:11.530470 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ffcc78-9b67-46d1-a092-3ad6c2c23178 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:11.532414 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-67ffcc78-9b67-46d1-a092-3ad6c2c23178 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:39:11.552752 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1141/2279] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jul 15 19:39:11 2022] POST /identity/v3/users => generated 287 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:39:11.555379 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a490196c-137c-433f-95f3-0767dd821bb6 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:39:11.566156 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a490196c-137c-433f-95f3-0767dd821bb6 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:39:11.571703 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6276369d-b6e9-4edf-979d-1e7e347934a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:39:11.572841 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6276369d-b6e9-4edf-979d-1e7e347934a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:11.573349 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6276369d-b6e9-4edf-979d-1e7e347934a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:11.574105 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6276369d-b6e9-4edf-979d-1e7e347934a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:11.575361 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6276369d-b6e9-4edf-979d-1e7e347934a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:39:11.578486 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6276369d-b6e9-4edf-979d-1e7e347934a2 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:39:11.594076 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1142/2280] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:39:11 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 19:39:11.594791 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6276369d-b6e9-4edf-979d-1e7e347934a2 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:39:11.605842 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6276369d-b6e9-4edf-979d-1e7e347934a2 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:39:11.611772 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe635f2-d24f-4704-a57b-5c13f784eda3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:39:11.612454 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe635f2-d24f-4704-a57b-5c13f784eda3 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:11.612752 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe635f2-d24f-4704-a57b-5c13f784eda3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:11.613028 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe635f2-d24f-4704-a57b-5c13f784eda3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:11.613880 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe635f2-d24f-4704-a57b-5c13f784eda3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:39:11.615557 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe635f2-d24f-4704-a57b-5c13f784eda3 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:39:11.617897 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ffcc78-9b67-46d1-a092-3ad6c2c23178 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:39:11.619457 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ffcc78-9b67-46d1-a092-3ad6c2c23178 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:39:11.627506 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1143/2281] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:39:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 19:39:11.629706 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1139/2282] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:39:11 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 19:39:11.631431 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe635f2-d24f-4704-a57b-5c13f784eda3 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:39:11.642123 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe635f2-d24f-4704-a57b-5c13f784eda3 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:39:11.646981 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc76553f-889a-4735-8cef-ccb6a5bbf900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:39:11.647861 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc76553f-889a-4735-8cef-ccb6a5bbf900 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:11.648146 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc76553f-889a-4735-8cef-ccb6a5bbf900 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:11.648430 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc76553f-889a-4735-8cef-ccb6a5bbf900 admin admin] PATH_INFO: `/v3/projects/6eb1bfa9f6044b4f95ad66e211e6c78b/users/a6f2e3bef31f43048d46d23a67fbf5ad/roles/28763c6e70094116b3a4b2895895fa58` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:11.664359 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc76553f-889a-4735-8cef-ccb6a5bbf900 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6eb1bfa9f6044b4f95ad66e211e6c78b, user_id=a6f2e3bef31f43048d46d23a67fbf5ad, role_id=28763c6e70094116b3a4b2895895fa58)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:39:11.667042 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc76553f-889a-4735-8cef-ccb6a5bbf900 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:39:11.735016 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1144/2283] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:39:11 2022] PUT /identity/v3/projects/6eb1bfa9f6044b4f95ad66e211e6c78b/users/a6f2e3bef31f43048d46d23a67fbf5ad/roles/28763c6e70094116b3a4b2895895fa58 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:39:11.746832 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16b8cc6-d9d4-4c5b-bfcd-bb7420306896 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:11.747159 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16b8cc6-d9d4-4c5b-bfcd-bb7420306896 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:11.747388 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16b8cc6-d9d4-4c5b-bfcd-bb7420306896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:11.776390 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-d16b8cc6-d9d4-4c5b-bfcd-bb7420306896 None None] MFA Rules not processed for user `a6f2e3bef31f43048d46d23a67fbf5ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:39:11.777588 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d16b8cc6-d9d4-4c5b-bfcd-bb7420306896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:39:11.846390 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1140/2284] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:39:11 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:39:11.857588 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc76553f-889a-4735-8cef-ccb6a5bbf900 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:39:11.859842 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-bc76553f-889a-4735-8cef-ccb6a5bbf900 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:39:11.916313 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc76553f-889a-4735-8cef-ccb6a5bbf900 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:39:11.919920 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a034b27e-e615-41ad-aa46-c95d9b74e853 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86c2df3d378490ca7ea497a73c19d40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:39:11.920587 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a034b27e-e615-41ad-aa46-c95d9b74e853 service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:11.920870 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a034b27e-e615-41ad-aa46-c95d9b74e853 service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:11.921144 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a034b27e-e615-41ad-aa46-c95d9b74e853 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:11.929447 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a034b27e-e615-41ad-aa46-c95d9b74e853 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:39:11.930923 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a034b27e-e615-41ad-aa46-c95d9b74e853 service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:39:11.940415 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1145/2285] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:39:11 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:39:12.620665 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:39:12.895198 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:39:13.969151 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:39:13.997771 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:39:14.177813 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:39:14.269067 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:39:14.449107 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:39:14.523407 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:39:14.898305 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:39:15.077711 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:39:15.706378 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:39:17.975967 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16b8cc6-d9d4-4c5b-bfcd-bb7420306896 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:39:17.977419 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d16b8cc6-d9d4-4c5b-bfcd-bb7420306896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:39:18.023377 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:39:18.051071 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16b8cc6-d9d4-4c5b-bfcd-bb7420306896 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:39:18.055733 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac0d43e-98e0-4f29-a496-a7e62cf25c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:39:18.056630 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac0d43e-98e0-4f29-a496-a7e62cf25c59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:18.056916 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac0d43e-98e0-4f29-a496-a7e62cf25c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:18.057193 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac0d43e-98e0-4f29-a496-a7e62cf25c59 admin admin] PATH_INFO: `/v3/users/49fac8536563457ba7e48446b2d62708` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:18.058431 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac0d43e-98e0-4f29-a496-a7e62cf25c59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49fac8536563457ba7e48446b2d62708)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:39:18.060435 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac0d43e-98e0-4f29-a496-a7e62cf25c59 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:39:18.283015 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-5ac0d43e-98e0-4f29-a496-a7e62cf25c59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49fac8536563457ba7e48446b2d62708'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:18.294685 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-5ac0d43e-98e0-4f29-a496-a7e62cf25c59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49fac8536563457ba7e48446b2d62708'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:18.383264 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-5ac0d43e-98e0-4f29-a496-a7e62cf25c59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49fac8536563457ba7e48446b2d62708'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:18.383927 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-5ac0d43e-98e0-4f29-a496-a7e62cf25c59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49fac8536563457ba7e48446b2d62708'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:18.391920 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-5ac0d43e-98e0-4f29-a496-a7e62cf25c59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49fac8536563457ba7e48446b2d62708'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:18.397989 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1141/2286] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jul 15 19:39:17 2022] DELETE /identity/v3/users/49fac8536563457ba7e48446b2d62708 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:39:18.402613 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de2d86-91cc-46e1-ab98-d204cbd5d7ec None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:18.403163 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de2d86-91cc-46e1-ab98-d204cbd5d7ec None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:18.403163 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de2d86-91cc-46e1-ab98-d204cbd5d7ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:18.421096 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-d0de2d86-91cc-46e1-ab98-d204cbd5d7ec None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:39:18.422338 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-d0de2d86-91cc-46e1-ab98-d204cbd5d7ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:39:18.480000 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1146/2287] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:39:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:39:18.490609 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac0d43e-98e0-4f29-a496-a7e62cf25c59 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:39:18.492079 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-5ac0d43e-98e0-4f29-a496-a7e62cf25c59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:39:18.556291 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac0d43e-98e0-4f29-a496-a7e62cf25c59 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:39:18.560022 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d98d00-77a6-4a2e-a8b5-dda3138e142e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:39:18.560675 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d98d00-77a6-4a2e-a8b5-dda3138e142e service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:18.560962 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d98d00-77a6-4a2e-a8b5-dda3138e142e service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:18.561244 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d98d00-77a6-4a2e-a8b5-dda3138e142e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:18.571142 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d98d00-77a6-4a2e-a8b5-dda3138e142e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:39:18.572680 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d98d00-77a6-4a2e-a8b5-dda3138e142e service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:39:18.582183 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1142/2288] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:39:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:39:19.027460 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0de2d86-91cc-46e1-ab98-d204cbd5d7ec None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:39:19.028884 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-d0de2d86-91cc-46e1-ab98-d204cbd5d7ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:39:19.045748 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0de2d86-91cc-46e1-ab98-d204cbd5d7ec None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:39:19.052252 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79fcd31-4a94-4b44-98ab-99d9d916711e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:39:19.053166 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79fcd31-4a94-4b44-98ab-99d9d916711e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:19.053313 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79fcd31-4a94-4b44-98ab-99d9d916711e admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:19.053731 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79fcd31-4a94-4b44-98ab-99d9d916711e admin admin] PATH_INFO: `/v3/projects/6fc539080bff4995a1d6b24ef2abf0aa` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:19.054964 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79fcd31-4a94-4b44-98ab-99d9d916711e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6fc539080bff4995a1d6b24ef2abf0aa)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:39:19.056845 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79fcd31-4a94-4b44-98ab-99d9d916711e admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:39:19.189609 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e79fcd31-4a94-4b44-98ab-99d9d916711e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6fc539080bff4995a1d6b24ef2abf0aa', 'request_id': 'req-e79fcd31-4a94-4b44-98ab-99d9d916711e'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:19.262511 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e79fcd31-4a94-4b44-98ab-99d9d916711e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6fc539080bff4995a1d6b24ef2abf0aa', 'request_id': 'req-e79fcd31-4a94-4b44-98ab-99d9d916711e'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:19.269597 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e79fcd31-4a94-4b44-98ab-99d9d916711e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6fc539080bff4995a1d6b24ef2abf0aa', 'request_id': 'req-e79fcd31-4a94-4b44-98ab-99d9d916711e'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:19.280136 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e79fcd31-4a94-4b44-98ab-99d9d916711e admin admin] The token cache is being invalidate because project 6fc539080bff4995a1d6b24ef2abf0aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:39:19.280428 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e79fcd31-4a94-4b44-98ab-99d9d916711e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:19.281066 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e79fcd31-4a94-4b44-98ab-99d9d916711e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:19.283276 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1147/2289] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jul 15 19:39:19 2022] DELETE /identity/v3/projects/6fc539080bff4995a1d6b24ef2abf0aa => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:39:19.287270 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d98d00-77a6-4a2e-a8b5-dda3138e142e service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:39:19.296963 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d98d00-77a6-4a2e-a8b5-dda3138e142e service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:39:19.353033 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c96a27-7296-47a5-aa1b-f1f21b70aca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:39:19.353955 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c96a27-7296-47a5-aa1b-f1f21b70aca4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:19.354240 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c96a27-7296-47a5-aa1b-f1f21b70aca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:19.354513 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c96a27-7296-47a5-aa1b-f1f21b70aca4 admin admin] PATH_INFO: `/v3/users/b26293a1e8624ae591356ac3c02c47a4` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:19.355741 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c96a27-7296-47a5-aa1b-f1f21b70aca4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b26293a1e8624ae591356ac3c02c47a4)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:39:19.357652 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c96a27-7296-47a5-aa1b-f1f21b70aca4 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:39:19.359260 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:39:19.614090 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-62c96a27-7296-47a5-aa1b-f1f21b70aca4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b26293a1e8624ae591356ac3c02c47a4'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:19.625681 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-62c96a27-7296-47a5-aa1b-f1f21b70aca4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b26293a1e8624ae591356ac3c02c47a4'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:19.700122 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-62c96a27-7296-47a5-aa1b-f1f21b70aca4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b26293a1e8624ae591356ac3c02c47a4'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:19.700752 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-62c96a27-7296-47a5-aa1b-f1f21b70aca4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b26293a1e8624ae591356ac3c02c47a4'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:19.708190 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-62c96a27-7296-47a5-aa1b-f1f21b70aca4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b26293a1e8624ae591356ac3c02c47a4'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:19.714255 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1143/2290] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jul 15 19:39:19 2022] DELETE /identity/v3/users/b26293a1e8624ae591356ac3c02c47a4 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:39:20.192120 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79fcd31-4a94-4b44-98ab-99d9d916711e admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:39:20.193582 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-e79fcd31-4a94-4b44-98ab-99d9d916711e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:39:20.210938 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:39:20.219387 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79fcd31-4a94-4b44-98ab-99d9d916711e admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:39:20.278822 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286fa49d-46e7-4822-bea6-a12df181d52e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:39:20.279696 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286fa49d-46e7-4822-bea6-a12df181d52e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:20.279978 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286fa49d-46e7-4822-bea6-a12df181d52e admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:20.280257 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286fa49d-46e7-4822-bea6-a12df181d52e admin admin] PATH_INFO: `/v3/projects/ed93f6a2878240539aaae22e65014111` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:20.282056 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286fa49d-46e7-4822-bea6-a12df181d52e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed93f6a2878240539aaae22e65014111)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:39:20.284009 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286fa49d-46e7-4822-bea6-a12df181d52e admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:39:20.403727 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-286fa49d-46e7-4822-bea6-a12df181d52e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed93f6a2878240539aaae22e65014111', 'request_id': 'req-286fa49d-46e7-4822-bea6-a12df181d52e'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:20.548699 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-286fa49d-46e7-4822-bea6-a12df181d52e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed93f6a2878240539aaae22e65014111', 'request_id': 'req-286fa49d-46e7-4822-bea6-a12df181d52e'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:20.555748 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-286fa49d-46e7-4822-bea6-a12df181d52e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed93f6a2878240539aaae22e65014111', 'request_id': 'req-286fa49d-46e7-4822-bea6-a12df181d52e'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:20.565996 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-286fa49d-46e7-4822-bea6-a12df181d52e admin admin] The token cache is being invalidate because project ed93f6a2878240539aaae22e65014111 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:39:20.566283 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-286fa49d-46e7-4822-bea6-a12df181d52e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:20.566905 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-286fa49d-46e7-4822-bea6-a12df181d52e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:20.568472 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1148/2291] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jul 15 19:39:20 2022] DELETE /identity/v3/projects/ed93f6a2878240539aaae22e65014111 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:39:20.574369 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5886150d-f904-4f4e-81dc-603b057c9408 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:20.574879 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5886150d-f904-4f4e-81dc-603b057c9408 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:20.575062 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5886150d-f904-4f4e-81dc-603b057c9408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:20.593089 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-5886150d-f904-4f4e-81dc-603b057c9408 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:39:20.594959 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-5886150d-f904-4f4e-81dc-603b057c9408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:39:20.653221 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1144/2292] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:39:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:39:20.661544 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e792a1d-7c14-4e00-ba41-ab760647d380 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:20.661872 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e792a1d-7c14-4e00-ba41-ab760647d380 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:20.662153 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e792a1d-7c14-4e00-ba41-ab760647d380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:20.681384 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-8e792a1d-7c14-4e00-ba41-ab760647d380 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:39:20.682525 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-8e792a1d-7c14-4e00-ba41-ab760647d380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:39:20.699264 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1149/2293] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:39:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:39:20.701994 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5886150d-f904-4f4e-81dc-603b057c9408 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:39:20.713719 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5886150d-f904-4f4e-81dc-603b057c9408 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:39:20.718515 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a130142b-3e29-4e5a-a0b0-3db80081c32b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:39:20.719770 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a130142b-3e29-4e5a-a0b0-3db80081c32b admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:20.720101 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a130142b-3e29-4e5a-a0b0-3db80081c32b admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:20.720374 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a130142b-3e29-4e5a-a0b0-3db80081c32b admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:20.721102 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a130142b-3e29-4e5a-a0b0-3db80081c32b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:39:20.721413 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a130142b-3e29-4e5a-a0b0-3db80081c32b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:39:20.723006 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a130142b-3e29-4e5a-a0b0-3db80081c32b admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:39:20.736331 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1145/2294] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jul 15 19:39:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:39:20.740730 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e792a1d-7c14-4e00-ba41-ab760647d380 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:39:20.750743 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e792a1d-7c14-4e00-ba41-ab760647d380 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:39:20.755444 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054d0160-ae7f-4aa3-9e08-72580945cf4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:39:20.756094 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d0160-ae7f-4aa3-9e08-72580945cf4d admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:20.756632 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d0160-ae7f-4aa3-9e08-72580945cf4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:20.756906 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d0160-ae7f-4aa3-9e08-72580945cf4d admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:20.757609 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054d0160-ae7f-4aa3-9e08-72580945cf4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:39:20.759599 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054d0160-ae7f-4aa3-9e08-72580945cf4d admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:39:20.871455 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1150/2295] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jul 15 19:39:20 2022] POST /identity/v3/projects => generated 369 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:39:20.872111 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a130142b-3e29-4e5a-a0b0-3db80081c32b admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:39:20.883448 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a130142b-3e29-4e5a-a0b0-3db80081c32b admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:39:20.952349 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74952aef-7350-4632-bd42-0a0a02ff02ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:39:20.953204 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74952aef-7350-4632-bd42-0a0a02ff02ce admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:20.954222 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74952aef-7350-4632-bd42-0a0a02ff02ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:20.954570 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74952aef-7350-4632-bd42-0a0a02ff02ce admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:20.955353 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74952aef-7350-4632-bd42-0a0a02ff02ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:39:20.958458 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74952aef-7350-4632-bd42-0a0a02ff02ce admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:39:20.976693 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:39:21.067566 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1146/2296] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jul 15 19:39:20 2022] POST /identity/v3/users => generated 285 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:39:21.068738 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054d0160-ae7f-4aa3-9e08-72580945cf4d admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:39:21.080331 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054d0160-ae7f-4aa3-9e08-72580945cf4d admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:39:21.084980 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fcce1a-7716-4a8a-b3b8-3558c0f85702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:39:21.085646 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fcce1a-7716-4a8a-b3b8-3558c0f85702 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:21.085967 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fcce1a-7716-4a8a-b3b8-3558c0f85702 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:21.086242 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fcce1a-7716-4a8a-b3b8-3558c0f85702 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:21.087038 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fcce1a-7716-4a8a-b3b8-3558c0f85702 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:39:21.088698 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fcce1a-7716-4a8a-b3b8-3558c0f85702 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:39:21.099940 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1151/2297] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:39:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:39:21.104231 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74952aef-7350-4632-bd42-0a0a02ff02ce admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:39:21.114948 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74952aef-7350-4632-bd42-0a0a02ff02ce admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:39:21.119714 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634f4155-0276-491b-af99-6676951fbdfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:39:21.120434 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634f4155-0276-491b-af99-6676951fbdfb admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:21.120668 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634f4155-0276-491b-af99-6676951fbdfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:21.120948 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634f4155-0276-491b-af99-6676951fbdfb admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:21.121682 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634f4155-0276-491b-af99-6676951fbdfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:39:21.123316 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634f4155-0276-491b-af99-6676951fbdfb admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:39:21.134908 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1147/2298] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:39:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:39:21.138920 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fcce1a-7716-4a8a-b3b8-3558c0f85702 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:39:21.149550 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fcce1a-7716-4a8a-b3b8-3558c0f85702 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:39:21.154358 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49471053-9a4f-43ed-8d5d-b65f127830d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:39:21.155236 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49471053-9a4f-43ed-8d5d-b65f127830d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:21.155519 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49471053-9a4f-43ed-8d5d-b65f127830d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:21.155798 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49471053-9a4f-43ed-8d5d-b65f127830d3 admin admin] PATH_INFO: `/v3/projects/c563f0ce3d5645778e9d7037c972b684/users/4b52eff4b9884943a445b918c99e139c/roles/28763c6e70094116b3a4b2895895fa58` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:21.170621 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49471053-9a4f-43ed-8d5d-b65f127830d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c563f0ce3d5645778e9d7037c972b684, user_id=4b52eff4b9884943a445b918c99e139c, role_id=28763c6e70094116b3a4b2895895fa58)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:39:21.173268 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49471053-9a4f-43ed-8d5d-b65f127830d3 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:39:21.246981 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1152/2299] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:39:21 2022] PUT /identity/v3/projects/c563f0ce3d5645778e9d7037c972b684/users/4b52eff4b9884943a445b918c99e139c/roles/28763c6e70094116b3a4b2895895fa58 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:39:21.258922 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf635c7-6a2a-43d4-a996-ba9ad4c7d132 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:21.262126 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf635c7-6a2a-43d4-a996-ba9ad4c7d132 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:21.262126 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf635c7-6a2a-43d4-a996-ba9ad4c7d132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:21.286115 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-bbf635c7-6a2a-43d4-a996-ba9ad4c7d132 None None] MFA Rules not processed for user `4b52eff4b9884943a445b918c99e139c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:39:21.287337 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-bbf635c7-6a2a-43d4-a996-ba9ad4c7d132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:39:21.350108 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1148/2300] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:39:21 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:39:21.361909 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49471053-9a4f-43ed-8d5d-b65f127830d3 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:39:21.365315 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-49471053-9a4f-43ed-8d5d-b65f127830d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:39:21.396535 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:39:21.420904 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49471053-9a4f-43ed-8d5d-b65f127830d3 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:39:21.425179 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347f6969-f275-46e8-90f9-e9ebb17583b6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04b5b06b0015450a95a28bbcc757cc8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:39:21.425849 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347f6969-f275-46e8-90f9-e9ebb17583b6 service glance] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:21.426132 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347f6969-f275-46e8-90f9-e9ebb17583b6 service glance] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:21.426404 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347f6969-f275-46e8-90f9-e9ebb17583b6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:21.434688 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347f6969-f275-46e8-90f9-e9ebb17583b6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:39:21.436123 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347f6969-f275-46e8-90f9-e9ebb17583b6 service glance] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:39:21.445638 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1153/2301] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:39:21 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:39:29.208823 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf635c7-6a2a-43d4-a996-ba9ad4c7d132 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:39:29.210264 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-bbf635c7-6a2a-43d4-a996-ba9ad4c7d132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:39:29.280990 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf635c7-6a2a-43d4-a996-ba9ad4c7d132 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:39:29.288716 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086e53f4-3265-429b-8c6e-2022a9c7169d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:39:29.289616 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086e53f4-3265-429b-8c6e-2022a9c7169d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:29.289933 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086e53f4-3265-429b-8c6e-2022a9c7169d admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:29.290213 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086e53f4-3265-429b-8c6e-2022a9c7169d admin admin] PATH_INFO: `/v3/users/a6f2e3bef31f43048d46d23a67fbf5ad` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:29.291430 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086e53f4-3265-429b-8c6e-2022a9c7169d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6f2e3bef31f43048d46d23a67fbf5ad)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:39:29.293445 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086e53f4-3265-429b-8c6e-2022a9c7169d admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:39:29.587476 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-086e53f4-3265-429b-8c6e-2022a9c7169d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6f2e3bef31f43048d46d23a67fbf5ad'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:29.599843 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-086e53f4-3265-429b-8c6e-2022a9c7169d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6f2e3bef31f43048d46d23a67fbf5ad'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:29.680221 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-086e53f4-3265-429b-8c6e-2022a9c7169d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6f2e3bef31f43048d46d23a67fbf5ad'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:29.680912 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-086e53f4-3265-429b-8c6e-2022a9c7169d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6f2e3bef31f43048d46d23a67fbf5ad'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:29.686914 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:39:29.690296 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-086e53f4-3265-429b-8c6e-2022a9c7169d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6f2e3bef31f43048d46d23a67fbf5ad'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:29.696312 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1149/2302] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jul 15 19:39:29 2022] DELETE /identity/v3/users/a6f2e3bef31f43048d46d23a67fbf5ad => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:39:29.701226 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04092365-bd4a-408d-8a67-b1c22049a949 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:29.701692 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04092365-bd4a-408d-8a67-b1c22049a949 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:29.701831 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04092365-bd4a-408d-8a67-b1c22049a949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:29.719133 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-04092365-bd4a-408d-8a67-b1c22049a949 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:39:29.720351 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-04092365-bd4a-408d-8a67-b1c22049a949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:39:29.783974 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1154/2303] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:39:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:39:29.796708 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086e53f4-3265-429b-8c6e-2022a9c7169d admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:39:29.797455 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-086e53f4-3265-429b-8c6e-2022a9c7169d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:39:29.858230 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086e53f4-3265-429b-8c6e-2022a9c7169d admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:39:29.862492 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f515aee-4791-42e1-82cc-26949c486317 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:39:29.863174 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f515aee-4791-42e1-82cc-26949c486317 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:29.863592 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f515aee-4791-42e1-82cc-26949c486317 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:29.863898 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f515aee-4791-42e1-82cc-26949c486317 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:29.873709 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f515aee-4791-42e1-82cc-26949c486317 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:39:29.875174 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f515aee-4791-42e1-82cc-26949c486317 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:39:29.885141 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1150/2304] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:39:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 19:39:30.379687 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04092365-bd4a-408d-8a67-b1c22049a949 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:39:30.381506 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-04092365-bd4a-408d-8a67-b1c22049a949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:39:30.397400 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04092365-bd4a-408d-8a67-b1c22049a949 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:39:30.402431 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22d0cad-b3d0-437a-ab3e-5be06ee7f05d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:39:30.403393 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22d0cad-b3d0-437a-ab3e-5be06ee7f05d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:30.403763 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22d0cad-b3d0-437a-ab3e-5be06ee7f05d admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:30.404114 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22d0cad-b3d0-437a-ab3e-5be06ee7f05d admin admin] PATH_INFO: `/v3/projects/6eb1bfa9f6044b4f95ad66e211e6c78b` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:30.405449 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22d0cad-b3d0-437a-ab3e-5be06ee7f05d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6eb1bfa9f6044b4f95ad66e211e6c78b)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:39:30.407598 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22d0cad-b3d0-437a-ab3e-5be06ee7f05d admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:39:30.529385 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e22d0cad-b3d0-437a-ab3e-5be06ee7f05d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6eb1bfa9f6044b4f95ad66e211e6c78b', 'request_id': 'req-e22d0cad-b3d0-437a-ab3e-5be06ee7f05d'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:30.588670 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e22d0cad-b3d0-437a-ab3e-5be06ee7f05d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6eb1bfa9f6044b4f95ad66e211e6c78b', 'request_id': 'req-e22d0cad-b3d0-437a-ab3e-5be06ee7f05d'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:30.595553 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e22d0cad-b3d0-437a-ab3e-5be06ee7f05d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6eb1bfa9f6044b4f95ad66e211e6c78b', 'request_id': 'req-e22d0cad-b3d0-437a-ab3e-5be06ee7f05d'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:30.606277 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e22d0cad-b3d0-437a-ab3e-5be06ee7f05d admin admin] The token cache is being invalidate because project 6eb1bfa9f6044b4f95ad66e211e6c78b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:39:30.606564 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e22d0cad-b3d0-437a-ab3e-5be06ee7f05d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:30.607316 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e22d0cad-b3d0-437a-ab3e-5be06ee7f05d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:39:30.608872 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1155/2305] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jul 15 19:39:30 2022] DELETE /identity/v3/projects/6eb1bfa9f6044b4f95ad66e211e6c78b => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:39:39.888143 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=29 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:39:40.603011 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=67,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:39:44.199628 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f515aee-4791-42e1-82cc-26949c486317 service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:39:44.201071 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-3f515aee-4791-42e1-82cc-26949c486317 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:39:44.271130 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f515aee-4791-42e1-82cc-26949c486317 service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:39:44.336434 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1771fa-0bc8-42a3-8c5b-937dbd923779 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52a0fce231ba4785a685ac9898d07503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:39:44.337098 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1771fa-0bc8-42a3-8c5b-937dbd923779 service placement] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:44.337379 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1771fa-0bc8-42a3-8c5b-937dbd923779 service placement] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:44.337656 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1771fa-0bc8-42a3-8c5b-937dbd923779 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:44.339548 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0a1771fa-0bc8-42a3-8c5b-937dbd923779 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:39:44.350951 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1771fa-0bc8-42a3-8c5b-937dbd923779 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:39:44.352591 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1771fa-0bc8-42a3-8c5b-937dbd923779 service placement] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:39:44.362442 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1151/2306] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:39:44 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:39:45.129403 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0992134-9828-4be2-8497-06b66eccc731 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:45.129941 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0992134-9828-4be2-8497-06b66eccc731 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:45.129941 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0992134-9828-4be2-8497-06b66eccc731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:45.147899 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-d0992134-9828-4be2-8497-06b66eccc731 None None] MFA Rules not processed for user `41d65200f05f4096823ecc0229b1ce3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:39:45.149110 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-d0992134-9828-4be2-8497-06b66eccc731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:39:45.204820 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1156/2307] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:39:45 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:39:54.358731 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=58 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:39:55.197294 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=25 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:40:13.405005 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1771fa-0bc8-42a3-8c5b-937dbd923779 service placement] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:40:13.411108 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0a1771fa-0bc8-42a3-8c5b-937dbd923779 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:40:13.477364 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1771fa-0bc8-42a3-8c5b-937dbd923779 service placement] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:40:13.483117 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f86c7b-b6ab-4b08-b61d-f25d1ce4d8f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:40:13.484001 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f86c7b-b6ab-4b08-b61d-f25d1ce4d8f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:40:13.484355 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f86c7b-b6ab-4b08-b61d-f25d1ce4d8f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:40:13.484639 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f86c7b-b6ab-4b08-b61d-f25d1ce4d8f6 admin admin] PATH_INFO: `/v3/users/e8596191b01c45de99a09e400c3677cb` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:40:13.485899 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f86c7b-b6ab-4b08-b61d-f25d1ce4d8f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8596191b01c45de99a09e400c3677cb)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:40:13.488131 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f86c7b-b6ab-4b08-b61d-f25d1ce4d8f6 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:40:13.827553 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d5f86c7b-b6ab-4b08-b61d-f25d1ce4d8f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8596191b01c45de99a09e400c3677cb'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:40:13.839591 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d5f86c7b-b6ab-4b08-b61d-f25d1ce4d8f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8596191b01c45de99a09e400c3677cb'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:40:13.938871 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d5f86c7b-b6ab-4b08-b61d-f25d1ce4d8f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8596191b01c45de99a09e400c3677cb'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:40:13.938871 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d5f86c7b-b6ab-4b08-b61d-f25d1ce4d8f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8596191b01c45de99a09e400c3677cb'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:40:13.942826 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d5f86c7b-b6ab-4b08-b61d-f25d1ce4d8f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8596191b01c45de99a09e400c3677cb'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:40:13.949429 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1152/2308] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jul 15 19:40:13 2022] DELETE /identity/v3/users/e8596191b01c45de99a09e400c3677cb => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:40:13.954669 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bedd210-82ab-4edc-87e5-fbf4fa0f0626 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:40:13.955204 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bedd210-82ab-4edc-87e5-fbf4fa0f0626 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:40:13.955204 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bedd210-82ab-4edc-87e5-fbf4fa0f0626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:40:13.973536 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-5bedd210-82ab-4edc-87e5-fbf4fa0f0626 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:40:13.974817 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-5bedd210-82ab-4edc-87e5-fbf4fa0f0626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:40:14.041533 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1157/2309] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:40:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:40:14.052718 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f86c7b-b6ab-4b08-b61d-f25d1ce4d8f6 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:40:14.054219 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d5f86c7b-b6ab-4b08-b61d-f25d1ce4d8f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:40:14.126610 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f86c7b-b6ab-4b08-b61d-f25d1ce4d8f6 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:40:14.131674 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ef66de-18c2-4e78-9bf1-4a57be0bb824 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:40:14.131674 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ef66de-18c2-4e78-9bf1-4a57be0bb824 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:40:14.131674 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ef66de-18c2-4e78-9bf1-4a57be0bb824 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:40:14.132004 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ef66de-18c2-4e78-9bf1-4a57be0bb824 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:40:14.143093 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ef66de-18c2-4e78-9bf1-4a57be0bb824 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:40:14.144687 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ef66de-18c2-4e78-9bf1-4a57be0bb824 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:40:14.156399 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1153/2310] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:40:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:40:14.709408 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bedd210-82ab-4edc-87e5-fbf4fa0f0626 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:40:14.710889 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-5bedd210-82ab-4edc-87e5-fbf4fa0f0626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:40:14.723828 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bedd210-82ab-4edc-87e5-fbf4fa0f0626 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:40:14.728720 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967f4c48-dbe6-4ec0-8232-2000956e3223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:40:14.729604 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967f4c48-dbe6-4ec0-8232-2000956e3223 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:40:14.729924 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967f4c48-dbe6-4ec0-8232-2000956e3223 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:40:14.730233 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967f4c48-dbe6-4ec0-8232-2000956e3223 admin admin] PATH_INFO: `/v3/projects/5c906616f9b044559e1c6db9015b97e1` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:40:14.731445 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967f4c48-dbe6-4ec0-8232-2000956e3223 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c906616f9b044559e1c6db9015b97e1)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:40:14.734115 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967f4c48-dbe6-4ec0-8232-2000956e3223 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:40:14.885315 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-967f4c48-dbe6-4ec0-8232-2000956e3223 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c906616f9b044559e1c6db9015b97e1', 'request_id': 'req-967f4c48-dbe6-4ec0-8232-2000956e3223'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:40:15.024289 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-967f4c48-dbe6-4ec0-8232-2000956e3223 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c906616f9b044559e1c6db9015b97e1', 'request_id': 'req-967f4c48-dbe6-4ec0-8232-2000956e3223'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:40:15.032318 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-967f4c48-dbe6-4ec0-8232-2000956e3223 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c906616f9b044559e1c6db9015b97e1', 'request_id': 'req-967f4c48-dbe6-4ec0-8232-2000956e3223'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:40:15.047158 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-967f4c48-dbe6-4ec0-8232-2000956e3223 admin admin] The token cache is being invalidate because project 5c906616f9b044559e1c6db9015b97e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:40:15.047497 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-967f4c48-dbe6-4ec0-8232-2000956e3223 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:40:15.049867 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-967f4c48-dbe6-4ec0-8232-2000956e3223 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:40:15.050210 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1158/2311] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jul 15 19:40:14 2022] DELETE /identity/v3/projects/5c906616f9b044559e1c6db9015b97e1 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:40:15.056299 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c9794d-b817-4e34-801c-43feb3cb2ab0 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:40:15.056784 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c9794d-b817-4e34-801c-43feb3cb2ab0 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:40:15.056880 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c9794d-b817-4e34-801c-43feb3cb2ab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:40:15.074676 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-17c9794d-b817-4e34-801c-43feb3cb2ab0 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:40:15.075877 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-17c9794d-b817-4e34-801c-43feb3cb2ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:40:15.093975 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:40:15.135156 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1154/2312] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:40:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:40:15.144080 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b529838b-aa42-4f29-87e2-0fb62c7551f8 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:40:15.144375 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b529838b-aa42-4f29-87e2-0fb62c7551f8 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:40:15.144657 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b529838b-aa42-4f29-87e2-0fb62c7551f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:40:15.163741 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-b529838b-aa42-4f29-87e2-0fb62c7551f8 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:40:15.164957 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-b529838b-aa42-4f29-87e2-0fb62c7551f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:40:15.173528 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1159/2313] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:40:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:40:15.177684 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c9794d-b817-4e34-801c-43feb3cb2ab0 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:40:15.187452 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c9794d-b817-4e34-801c-43feb3cb2ab0 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:40:15.192280 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f339c3a4-023d-4101-bd0a-c6fc72904886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:40:15.192938 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f339c3a4-023d-4101-bd0a-c6fc72904886 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:40:15.193223 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f339c3a4-023d-4101-bd0a-c6fc72904886 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:40:15.193540 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f339c3a4-023d-4101-bd0a-c6fc72904886 admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:40:15.194402 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f339c3a4-023d-4101-bd0a-c6fc72904886 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:40:15.194712 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f339c3a4-023d-4101-bd0a-c6fc72904886 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:40:15.196383 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f339c3a4-023d-4101-bd0a-c6fc72904886 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:40:15.209213 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1155/2314] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jul 15 19:40:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:40:15.213520 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b529838b-aa42-4f29-87e2-0fb62c7551f8 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:40:15.223217 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b529838b-aa42-4f29-87e2-0fb62c7551f8 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:40:15.231016 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3bccd9-7baf-4b63-8b78-a86d46f6e8bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:40:15.231016 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3bccd9-7baf-4b63-8b78-a86d46f6e8bd admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:40:15.231016 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3bccd9-7baf-4b63-8b78-a86d46f6e8bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:40:15.231016 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3bccd9-7baf-4b63-8b78-a86d46f6e8bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:40:15.231369 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3bccd9-7baf-4b63-8b78-a86d46f6e8bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:40:15.232786 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3bccd9-7baf-4b63-8b78-a86d46f6e8bd admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:40:15.368586 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1160/2315] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jul 15 19:40:15 2022] POST /identity/v3/projects => generated 391 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:40:15.370112 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f339c3a4-023d-4101-bd0a-c6fc72904886 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:40:15.382237 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f339c3a4-023d-4101-bd0a-c6fc72904886 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:40:15.454966 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd159b3-2755-4b2f-813f-09f9d56616f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:40:15.455817 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd159b3-2755-4b2f-813f-09f9d56616f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:40:15.456052 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd159b3-2755-4b2f-813f-09f9d56616f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:40:15.456341 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd159b3-2755-4b2f-813f-09f9d56616f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:40:15.457706 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd159b3-2755-4b2f-813f-09f9d56616f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:40:15.459120 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd159b3-2755-4b2f-813f-09f9d56616f7 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:40:15.710106 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1156/2316] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jul 15 19:40:15 2022] POST /identity/v3/users => generated 296 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:40:15.719854 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3bccd9-7baf-4b63-8b78-a86d46f6e8bd admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:40:15.740422 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3bccd9-7baf-4b63-8b78-a86d46f6e8bd admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:40:15.746210 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2196ab-7654-4381-8848-beb714a05086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:40:15.746968 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2196ab-7654-4381-8848-beb714a05086 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:40:15.747252 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2196ab-7654-4381-8848-beb714a05086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:40:15.747534 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2196ab-7654-4381-8848-beb714a05086 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:40:15.749035 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2196ab-7654-4381-8848-beb714a05086 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:40:15.750771 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2196ab-7654-4381-8848-beb714a05086 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:40:15.767694 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1161/2317] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:40:15 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:40:15.769897 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd159b3-2755-4b2f-813f-09f9d56616f7 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:40:15.787110 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd159b3-2755-4b2f-813f-09f9d56616f7 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:40:15.793226 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979f3231-2f26-4da6-b5fc-6227ec309438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:40:15.794272 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979f3231-2f26-4da6-b5fc-6227ec309438 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:40:15.794558 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979f3231-2f26-4da6-b5fc-6227ec309438 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:40:15.795381 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979f3231-2f26-4da6-b5fc-6227ec309438 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:40:15.796127 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979f3231-2f26-4da6-b5fc-6227ec309438 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:40:15.799032 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979f3231-2f26-4da6-b5fc-6227ec309438 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:40:15.816481 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1157/2318] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:40:15 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:40:15.821612 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2196ab-7654-4381-8848-beb714a05086 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:40:15.835373 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2196ab-7654-4381-8848-beb714a05086 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:40:15.840864 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdcefa4-f5b8-4414-b4fe-4a6cc6797b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:40:15.842113 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdcefa4-f5b8-4414-b4fe-4a6cc6797b10 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:40:15.842307 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdcefa4-f5b8-4414-b4fe-4a6cc6797b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:40:15.842549 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdcefa4-f5b8-4414-b4fe-4a6cc6797b10 admin admin] PATH_INFO: `/v3/projects/5f679df434534f34bd6ae50dc28ffb0b/users/7dfcd1bd64854895ac45d36c63455181/roles/28763c6e70094116b3a4b2895895fa58` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:40:15.868914 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdcefa4-f5b8-4414-b4fe-4a6cc6797b10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f679df434534f34bd6ae50dc28ffb0b, user_id=7dfcd1bd64854895ac45d36c63455181, role_id=28763c6e70094116b3a4b2895895fa58)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:40:15.871619 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdcefa4-f5b8-4414-b4fe-4a6cc6797b10 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:40:15.939542 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979f3231-2f26-4da6-b5fc-6227ec309438 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:40:15.941286 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-979f3231-2f26-4da6-b5fc-6227ec309438 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:40:15.988880 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1162/2319] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:40:15 2022] PUT /identity/v3/projects/5f679df434534f34bd6ae50dc28ffb0b/users/7dfcd1bd64854895ac45d36c63455181/roles/28763c6e70094116b3a4b2895895fa58 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:40:16.000030 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d912666-356e-4221-9aa3-a8a79a99af59 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:40:16.000321 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d912666-356e-4221-9aa3-a8a79a99af59 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:40:16.000623 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d912666-356e-4221-9aa3-a8a79a99af59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:40:16.001487 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979f3231-2f26-4da6-b5fc-6227ec309438 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:40:16.027452 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-3d912666-356e-4221-9aa3-a8a79a99af59 None None] MFA Rules not processed for user `7dfcd1bd64854895ac45d36c63455181`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:40:16.028751 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-3d912666-356e-4221-9aa3-a8a79a99af59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:40:16.037172 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:40:16.046525 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:40:16.054548 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a77caf4-21ca-436f-963c-29b78dfaf856 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:40:16.055520 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a77caf4-21ca-436f-963c-29b78dfaf856 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:40:16.056063 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a77caf4-21ca-436f-963c-29b78dfaf856 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:40:16.056523 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a77caf4-21ca-436f-963c-29b78dfaf856 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:40:16.059124 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-3a77caf4-21ca-436f-963c-29b78dfaf856 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:40:16.096041 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1163/2320] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:40:15 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:40:16.106981 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d912666-356e-4221-9aa3-a8a79a99af59 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:40:16.108913 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-3d912666-356e-4221-9aa3-a8a79a99af59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:40:16.126409 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a77caf4-21ca-436f-963c-29b78dfaf856 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:40:16.127855 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a77caf4-21ca-436f-963c-29b78dfaf856 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:40:16.137676 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1158/2321] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:40:15 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 19:40:16.167709 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d912666-356e-4221-9aa3-a8a79a99af59 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:40:16.171762 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7a4efd-39a0-4cfe-ad8f-6ce3792fca61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86c2df3d378490ca7ea497a73c19d40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:40:16.172415 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7a4efd-39a0-4cfe-ad8f-6ce3792fca61 service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:40:16.172699 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7a4efd-39a0-4cfe-ad8f-6ce3792fca61 service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:40:16.172978 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7a4efd-39a0-4cfe-ad8f-6ce3792fca61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:40:16.181810 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7a4efd-39a0-4cfe-ad8f-6ce3792fca61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:40:16.183390 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7a4efd-39a0-4cfe-ad8f-6ce3792fca61 service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:40:16.193097 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1164/2322] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:40:16 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 19:40:26.473173 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=30 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:40:26.622195 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=49 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:40:29.539533 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55606e0-2f7f-4915-b1ec-6cb425521af8 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:40:29.540356 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55606e0-2f7f-4915-b1ec-6cb425521af8 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:40:29.540988 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55606e0-2f7f-4915-b1ec-6cb425521af8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:40:29.563816 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-f55606e0-2f7f-4915-b1ec-6cb425521af8 None None] MFA Rules not processed for user `3d893c14620b4a129842062f14a68a78`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:40:29.565035 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-f55606e0-2f7f-4915-b1ec-6cb425521af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:40:29.625055 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1159/2323] 10.222.0.33 () {58 vars in 1009 bytes} [Fri Jul 15 19:40:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:40:29.793750 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7a4efd-39a0-4cfe-ad8f-6ce3792fca61 service cinder] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:40:29.795794 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-ae7a4efd-39a0-4cfe-ad8f-6ce3792fca61 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:40:29.865345 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7a4efd-39a0-4cfe-ad8f-6ce3792fca61 service cinder] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:40:29.870405 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ee0b1c-f4aa-4498-91ae-d001ad03b474 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef76f7fe39a3499aaeebf85c176804f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:40:29.871115 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee0b1c-f4aa-4498-91ae-d001ad03b474 service swift] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:40:29.871339 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee0b1c-f4aa-4498-91ae-d001ad03b474 service swift] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:40:29.871617 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee0b1c-f4aa-4498-91ae-d001ad03b474 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:40:29.879846 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ee0b1c-f4aa-4498-91ae-d001ad03b474 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:40:29.881424 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ee0b1c-f4aa-4498-91ae-d001ad03b474 service swift] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:40:29.889483 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1165/2324] 10.222.0.33 () {60 vars in 1384 bytes} [Fri Jul 15 19:40:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 19:40:30.031935 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55606e0-2f7f-4915-b1ec-6cb425521af8 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:40:30.033466 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-f55606e0-2f7f-4915-b1ec-6cb425521af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:40:30.046387 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55606e0-2f7f-4915-b1ec-6cb425521af8 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:40:30.050562 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe1d6e-cb63-4dfb-a335-a359df0d2242 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:40:30.056268 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe1d6e-cb63-4dfb-a335-a359df0d2242 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:40:30.056268 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe1d6e-cb63-4dfb-a335-a359df0d2242 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:40:30.056268 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe1d6e-cb63-4dfb-a335-a359df0d2242 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:40:30.056268 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-3cbe1d6e-cb63-4dfb-a335-a359df0d2242 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:40:30.144787 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbe1d6e-cb63-4dfb-a335-a359df0d2242 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:40:30.154299 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbe1d6e-cb63-4dfb-a335-a359df0d2242 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:40:30.159245 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1160/2325] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:40:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:40:32.014432 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ee0b1c-f4aa-4498-91ae-d001ad03b474 service swift] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:40:32.025865 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ee0b1c-f4aa-4498-91ae-d001ad03b474 service swift] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:40:32.029828 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b597b313-e22a-4338-8e40-12e79d47de63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:40:32.030733 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b597b313-e22a-4338-8e40-12e79d47de63 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:40:32.031094 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b597b313-e22a-4338-8e40-12e79d47de63 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:40:32.031532 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b597b313-e22a-4338-8e40-12e79d47de63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:40:32.034804 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-b597b313-e22a-4338-8e40-12e79d47de63 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:40:32.050512 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b597b313-e22a-4338-8e40-12e79d47de63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:40:32.052302 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b597b313-e22a-4338-8e40-12e79d47de63 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:40:32.066029 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1166/2326] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:40:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:40:38.832679 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe1d6e-cb63-4dfb-a335-a359df0d2242 service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:40:38.834247 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-3cbe1d6e-cb63-4dfb-a335-a359df0d2242 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:40:38.912819 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe1d6e-cb63-4dfb-a335-a359df0d2242 service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:40:38.918314 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a514b2-20da-40a4-b326-578eedcfe5fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:40:38.919377 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a514b2-20da-40a4-b326-578eedcfe5fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:40:38.919669 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a514b2-20da-40a4-b326-578eedcfe5fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:40:38.920063 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a514b2-20da-40a4-b326-578eedcfe5fb admin admin] PATH_INFO: `/v3/users/4b52eff4b9884943a445b918c99e139c` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:40:38.921340 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a514b2-20da-40a4-b326-578eedcfe5fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b52eff4b9884943a445b918c99e139c)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:40:38.923331 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a514b2-20da-40a4-b326-578eedcfe5fb admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:40:39.179867 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:40:39.193312 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-77a514b2-20da-40a4-b326-578eedcfe5fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b52eff4b9884943a445b918c99e139c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:40:39.211042 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-77a514b2-20da-40a4-b326-578eedcfe5fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b52eff4b9884943a445b918c99e139c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:40:39.332358 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-77a514b2-20da-40a4-b326-578eedcfe5fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b52eff4b9884943a445b918c99e139c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:40:39.337209 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-77a514b2-20da-40a4-b326-578eedcfe5fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b52eff4b9884943a445b918c99e139c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:40:39.342559 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-77a514b2-20da-40a4-b326-578eedcfe5fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b52eff4b9884943a445b918c99e139c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:40:39.351261 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1161/2327] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jul 15 19:40:38 2022] DELETE /identity/v3/users/4b52eff4b9884943a445b918c99e139c => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:40:39.356118 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbf1400-1730-42ff-b32c-9fcb2ed82225 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:40:39.357108 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbf1400-1730-42ff-b32c-9fcb2ed82225 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:40:39.357561 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbf1400-1730-42ff-b32c-9fcb2ed82225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:40:39.380249 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-2dbf1400-1730-42ff-b32c-9fcb2ed82225 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:40:39.381700 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-2dbf1400-1730-42ff-b32c-9fcb2ed82225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:40:39.461669 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1167/2328] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:40:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:40:39.470920 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a514b2-20da-40a4-b326-578eedcfe5fb admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:40:39.473098 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-77a514b2-20da-40a4-b326-578eedcfe5fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:40:39.565393 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a514b2-20da-40a4-b326-578eedcfe5fb admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:40:39.571165 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbab1849-cd8f-4c0b-b179-cd98496bc297 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:40:39.571833 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab1849-cd8f-4c0b-b179-cd98496bc297 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:40:39.572117 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab1849-cd8f-4c0b-b179-cd98496bc297 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:40:39.572384 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab1849-cd8f-4c0b-b179-cd98496bc297 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:40:39.585289 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbab1849-cd8f-4c0b-b179-cd98496bc297 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:40:39.587945 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbab1849-cd8f-4c0b-b179-cd98496bc297 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:40:39.602060 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1162/2329] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:40:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:40:40.700427 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbf1400-1730-42ff-b32c-9fcb2ed82225 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:40:40.761176 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-2dbf1400-1730-42ff-b32c-9fcb2ed82225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:40:40.761176 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbf1400-1730-42ff-b32c-9fcb2ed82225 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:40:40.761176 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739eee66-27a2-42e9-9908-f626b52add39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:40:40.761899 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739eee66-27a2-42e9-9908-f626b52add39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:40:40.761899 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739eee66-27a2-42e9-9908-f626b52add39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:40:40.761899 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739eee66-27a2-42e9-9908-f626b52add39 admin admin] PATH_INFO: `/v3/projects/c563f0ce3d5645778e9d7037c972b684` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:40:40.761899 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739eee66-27a2-42e9-9908-f626b52add39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c563f0ce3d5645778e9d7037c972b684)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:40:40.761899 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739eee66-27a2-42e9-9908-f626b52add39 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:40:41.165365 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-739eee66-27a2-42e9-9908-f626b52add39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c563f0ce3d5645778e9d7037c972b684', 'request_id': 'req-739eee66-27a2-42e9-9908-f626b52add39'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:40:41.390018 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-739eee66-27a2-42e9-9908-f626b52add39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c563f0ce3d5645778e9d7037c972b684', 'request_id': 'req-739eee66-27a2-42e9-9908-f626b52add39'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:40:41.397025 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-739eee66-27a2-42e9-9908-f626b52add39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c563f0ce3d5645778e9d7037c972b684', 'request_id': 'req-739eee66-27a2-42e9-9908-f626b52add39'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:40:41.406207 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:40:41.410055 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-739eee66-27a2-42e9-9908-f626b52add39 admin admin] The token cache is being invalidate because project c563f0ce3d5645778e9d7037c972b684 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:40:41.410340 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-739eee66-27a2-42e9-9908-f626b52add39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:40:41.411039 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-739eee66-27a2-42e9-9908-f626b52add39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:40:41.412821 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1168/2330] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jul 15 19:40:40 2022] DELETE /identity/v3/projects/c563f0ce3d5645778e9d7037c972b684 => generated 0 bytes in 713 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:40:41.417723 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6332f725-f401-4d95-bc70-e7623c259eb9 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:40:41.418244 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6332f725-f401-4d95-bc70-e7623c259eb9 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:40:41.418341 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6332f725-f401-4d95-bc70-e7623c259eb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:40:41.437019 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-6332f725-f401-4d95-bc70-e7623c259eb9 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:40:41.438899 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-6332f725-f401-4d95-bc70-e7623c259eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:40:41.526217 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1163/2331] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:40:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:40:41.534106 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e276f-2a44-4418-9fa5-a101f5def247 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:40:41.534425 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e276f-2a44-4418-9fa5-a101f5def247 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:40:41.534712 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e276f-2a44-4418-9fa5-a101f5def247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:40:41.555088 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-ff2e276f-2a44-4418-9fa5-a101f5def247 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:40:41.556344 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-ff2e276f-2a44-4418-9fa5-a101f5def247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:40:41.569823 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1169/2332] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:40:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:40:41.570073 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6332f725-f401-4d95-bc70-e7623c259eb9 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:40:41.580346 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6332f725-f401-4d95-bc70-e7623c259eb9 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:40:41.585621 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db21da70-5d05-436a-9b8f-6062dc5e71eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:40:41.586421 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db21da70-5d05-436a-9b8f-6062dc5e71eb admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:40:41.586698 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db21da70-5d05-436a-9b8f-6062dc5e71eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:40:41.586970 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db21da70-5d05-436a-9b8f-6062dc5e71eb admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:40:41.587714 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db21da70-5d05-436a-9b8f-6062dc5e71eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:40:41.588020 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db21da70-5d05-436a-9b8f-6062dc5e71eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:40:41.589653 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db21da70-5d05-436a-9b8f-6062dc5e71eb admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:40:41.603546 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1164/2333] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jul 15 19:40:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:40:41.608034 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2e276f-2a44-4418-9fa5-a101f5def247 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:40:41.617872 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2e276f-2a44-4418-9fa5-a101f5def247 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:40:41.623418 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45874adb-9c60-42aa-bb88-9e11e3d8171e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:40:41.624092 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45874adb-9c60-42aa-bb88-9e11e3d8171e admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:40:41.624377 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45874adb-9c60-42aa-bb88-9e11e3d8171e admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:40:41.624664 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45874adb-9c60-42aa-bb88-9e11e3d8171e admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:40:41.625349 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45874adb-9c60-42aa-bb88-9e11e3d8171e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:40:41.627319 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45874adb-9c60-42aa-bb88-9e11e3d8171e admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:40:41.996091 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1170/2334] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jul 15 19:40:41 2022] POST /identity/v3/projects => generated 371 bytes in 389 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:40:42.000291 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db21da70-5d05-436a-9b8f-6062dc5e71eb admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:40:42.011150 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db21da70-5d05-436a-9b8f-6062dc5e71eb admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:40:42.049159 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:40:42.068753 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b294e220-54a3-4541-86f3-dcfc9b11cea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:40:42.069423 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b294e220-54a3-4541-86f3-dcfc9b11cea9 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:40:42.069704 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b294e220-54a3-4541-86f3-dcfc9b11cea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:40:42.070016 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b294e220-54a3-4541-86f3-dcfc9b11cea9 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:40:42.070714 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b294e220-54a3-4541-86f3-dcfc9b11cea9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:40:42.072665 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b294e220-54a3-4541-86f3-dcfc9b11cea9 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:40:42.165070 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1165/2335] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jul 15 19:40:41 2022] POST /identity/v3/users => generated 286 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:40:42.169017 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45874adb-9c60-42aa-bb88-9e11e3d8171e admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:40:42.179625 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45874adb-9c60-42aa-bb88-9e11e3d8171e admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:40:42.184100 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb66896-d14c-490c-9262-97b07a726495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:40:42.184759 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb66896-d14c-490c-9262-97b07a726495 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:40:42.185036 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb66896-d14c-490c-9262-97b07a726495 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:40:42.185307 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb66896-d14c-490c-9262-97b07a726495 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:40:42.186056 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb66896-d14c-490c-9262-97b07a726495 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:40:42.187639 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb66896-d14c-490c-9262-97b07a726495 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:40:42.198104 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1171/2336] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:40:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:40:42.207479 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b294e220-54a3-4541-86f3-dcfc9b11cea9 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:40:42.218434 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b294e220-54a3-4541-86f3-dcfc9b11cea9 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:40:42.223028 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413b5dd6-0b33-461d-af59-000b9b63f95a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:40:42.223728 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413b5dd6-0b33-461d-af59-000b9b63f95a admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:40:42.224013 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413b5dd6-0b33-461d-af59-000b9b63f95a admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:40:42.224285 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413b5dd6-0b33-461d-af59-000b9b63f95a admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:40:42.225008 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413b5dd6-0b33-461d-af59-000b9b63f95a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:40:42.226642 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413b5dd6-0b33-461d-af59-000b9b63f95a admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:40:42.237838 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1166/2337] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:40:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:40:42.242078 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb66896-d14c-490c-9262-97b07a726495 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:40:42.252070 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb66896-d14c-490c-9262-97b07a726495 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:40:42.258820 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ab66e4-ee06-439a-b38d-2c6732a88bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:40:42.259711 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ab66e4-ee06-439a-b38d-2c6732a88bd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:40:42.259994 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ab66e4-ee06-439a-b38d-2c6732a88bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:40:42.260271 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ab66e4-ee06-439a-b38d-2c6732a88bd3 admin admin] PATH_INFO: `/v3/projects/b29755b436734faf8822c03af7406b99/users/a009c766b98b48678e7676e2f55a5adc/roles/28763c6e70094116b3a4b2895895fa58` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:40:42.277998 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ab66e4-ee06-439a-b38d-2c6732a88bd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b29755b436734faf8822c03af7406b99, user_id=a009c766b98b48678e7676e2f55a5adc, role_id=28763c6e70094116b3a4b2895895fa58)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:40:42.281315 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ab66e4-ee06-439a-b38d-2c6732a88bd3 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:40:42.344566 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413b5dd6-0b33-461d-af59-000b9b63f95a admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:40:42.346405 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-413b5dd6-0b33-461d-af59-000b9b63f95a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:40:42.390065 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1172/2338] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:40:42 2022] PUT /identity/v3/projects/b29755b436734faf8822c03af7406b99/users/a009c766b98b48678e7676e2f55a5adc/roles/28763c6e70094116b3a4b2895895fa58 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:40:42.436021 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413b5dd6-0b33-461d-af59-000b9b63f95a admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:40:42.459666 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f255f67-0efa-4239-b280-9a63d98a4f21 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:40:42.459954 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f255f67-0efa-4239-b280-9a63d98a4f21 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:40:42.460221 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f255f67-0efa-4239-b280-9a63d98a4f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:40:42.488026 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-3f255f67-0efa-4239-b280-9a63d98a4f21 None None] MFA Rules not processed for user `a009c766b98b48678e7676e2f55a5adc`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:40:42.489509 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-3f255f67-0efa-4239-b280-9a63d98a4f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:40:42.506765 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc682ec3-4e19-4fb9-945d-3645173a3db5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7673a3b5c0324e07a9535ac70ed7a491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:40:42.507240 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc682ec3-4e19-4fb9-945d-3645173a3db5 service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:40:42.507527 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc682ec3-4e19-4fb9-945d-3645173a3db5 service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:40:42.507808 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc682ec3-4e19-4fb9-945d-3645173a3db5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:40:42.510320 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-fc682ec3-4e19-4fb9-945d-3645173a3db5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:40:42.557713 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1173/2339] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:40:42 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:40:42.569825 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:40:42.570110 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f255f67-0efa-4239-b280-9a63d98a4f21 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:40:42.571555 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-3f255f67-0efa-4239-b280-9a63d98a4f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:40:42.573377 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc682ec3-4e19-4fb9-945d-3645173a3db5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:40:42.575682 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc682ec3-4e19-4fb9-945d-3645173a3db5 service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:40:42.585400 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1167/2340] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:40:42 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 19:40:42.630695 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f255f67-0efa-4239-b280-9a63d98a4f21 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:40:42.634327 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c2c908-a962-4e57-aa82-504b5a7cf2ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86c2df3d378490ca7ea497a73c19d40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:40:42.634965 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c2c908-a962-4e57-aa82-504b5a7cf2ba service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:40:42.635245 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c2c908-a962-4e57-aa82-504b5a7cf2ba service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:40:42.635516 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c2c908-a962-4e57-aa82-504b5a7cf2ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:40:42.644926 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c2c908-a962-4e57-aa82-504b5a7cf2ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:40:42.646494 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c2c908-a962-4e57-aa82-504b5a7cf2ba service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:40:42.655674 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1174/2341] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:40:42 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 19:40:42.682135 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc682ec3-4e19-4fb9-945d-3645173a3db5 service nova] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:40:42.751848 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc682ec3-4e19-4fb9-945d-3645173a3db5 service nova] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:40:42.756809 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244918e9-dbde-41cc-83be-38d8cbd84a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:40:42.757690 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244918e9-dbde-41cc-83be-38d8cbd84a3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:40:42.758007 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244918e9-dbde-41cc-83be-38d8cbd84a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:40:42.758287 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244918e9-dbde-41cc-83be-38d8cbd84a3f admin admin] PATH_INFO: `/v3/users/a009c766b98b48678e7676e2f55a5adc` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:40:42.759507 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244918e9-dbde-41cc-83be-38d8cbd84a3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a009c766b98b48678e7676e2f55a5adc)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:40:42.761449 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244918e9-dbde-41cc-83be-38d8cbd84a3f admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:40:42.974029 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-244918e9-dbde-41cc-83be-38d8cbd84a3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a009c766b98b48678e7676e2f55a5adc'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:40:42.986151 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-244918e9-dbde-41cc-83be-38d8cbd84a3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a009c766b98b48678e7676e2f55a5adc'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:40:43.078522 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-244918e9-dbde-41cc-83be-38d8cbd84a3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a009c766b98b48678e7676e2f55a5adc'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:40:43.079131 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-244918e9-dbde-41cc-83be-38d8cbd84a3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a009c766b98b48678e7676e2f55a5adc'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:40:43.086554 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-244918e9-dbde-41cc-83be-38d8cbd84a3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a009c766b98b48678e7676e2f55a5adc'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:40:43.092571 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1168/2342] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jul 15 19:40:42 2022] DELETE /identity/v3/users/a009c766b98b48678e7676e2f55a5adc => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:40:43.096952 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93275d8-167d-43d4-91bb-d7978c9f1767 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:40:43.097457 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93275d8-167d-43d4-91bb-d7978c9f1767 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:40:43.097569 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93275d8-167d-43d4-91bb-d7978c9f1767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:40:43.114969 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-d93275d8-167d-43d4-91bb-d7978c9f1767 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:40:43.116162 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-d93275d8-167d-43d4-91bb-d7978c9f1767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:40:43.140433 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:40:43.180669 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1175/2343] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:40:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:40:43.188076 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244918e9-dbde-41cc-83be-38d8cbd84a3f admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:40:43.189508 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-244918e9-dbde-41cc-83be-38d8cbd84a3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:40:43.248219 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244918e9-dbde-41cc-83be-38d8cbd84a3f admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:40:43.251823 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a7a111-5e0c-4f35-b765-51f42d2f6975 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:40:43.252480 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a7a111-5e0c-4f35-b765-51f42d2f6975 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:40:43.252776 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a7a111-5e0c-4f35-b765-51f42d2f6975 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:40:43.253050 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a7a111-5e0c-4f35-b765-51f42d2f6975 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:40:43.263514 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a7a111-5e0c-4f35-b765-51f42d2f6975 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:40:43.264967 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a7a111-5e0c-4f35-b765-51f42d2f6975 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:40:43.275049 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1169/2344] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:40:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:40:43.826814 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93275d8-167d-43d4-91bb-d7978c9f1767 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:40:43.826814 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-d93275d8-167d-43d4-91bb-d7978c9f1767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:40:43.837150 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93275d8-167d-43d4-91bb-d7978c9f1767 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:40:43.842485 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8d621b-8177-4be6-b590-1909e9235353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:40:43.843367 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8d621b-8177-4be6-b590-1909e9235353 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:40:43.843649 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8d621b-8177-4be6-b590-1909e9235353 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:40:43.843924 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8d621b-8177-4be6-b590-1909e9235353 admin admin] PATH_INFO: `/v3/projects/b29755b436734faf8822c03af7406b99` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:40:43.845285 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8d621b-8177-4be6-b590-1909e9235353 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b29755b436734faf8822c03af7406b99)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:40:43.847233 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8d621b-8177-4be6-b590-1909e9235353 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:40:44.067696 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-ff8d621b-8177-4be6-b590-1909e9235353 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b29755b436734faf8822c03af7406b99', 'request_id': 'req-ff8d621b-8177-4be6-b590-1909e9235353'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:40:44.332002 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-ff8d621b-8177-4be6-b590-1909e9235353 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b29755b436734faf8822c03af7406b99', 'request_id': 'req-ff8d621b-8177-4be6-b590-1909e9235353'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:40:44.338697 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-ff8d621b-8177-4be6-b590-1909e9235353 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b29755b436734faf8822c03af7406b99', 'request_id': 'req-ff8d621b-8177-4be6-b590-1909e9235353'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:40:44.348900 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-ff8d621b-8177-4be6-b590-1909e9235353 admin admin] The token cache is being invalidate because project b29755b436734faf8822c03af7406b99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:40:44.349186 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-ff8d621b-8177-4be6-b590-1909e9235353 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:40:44.349854 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-ff8d621b-8177-4be6-b590-1909e9235353 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:40:44.351432 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1176/2345] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jul 15 19:40:43 2022] DELETE /identity/v3/projects/b29755b436734faf8822c03af7406b99 => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:40:46.963136 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a7a111-5e0c-4f35-b765-51f42d2f6975 service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:40:46.964616 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-c7a7a111-5e0c-4f35-b765-51f42d2f6975 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:40:46.984393 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:40:47.038787 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a7a111-5e0c-4f35-b765-51f42d2f6975 service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:40:47.106816 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa0c91b-9039-44fe-b09d-8377b9d616e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7673a3b5c0324e07a9535ac70ed7a491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:40:47.107396 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa0c91b-9039-44fe-b09d-8377b9d616e9 service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:40:47.107680 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa0c91b-9039-44fe-b09d-8377b9d616e9 service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:40:47.107957 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa0c91b-9039-44fe-b09d-8377b9d616e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:40:47.109796 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-7aa0c91b-9039-44fe-b09d-8377b9d616e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:40:47.167366 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa0c91b-9039-44fe-b09d-8377b9d616e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:40:47.169568 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa0c91b-9039-44fe-b09d-8377b9d616e9 service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:40:47.178843 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1170/2346] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:40:46 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:40:50.054806 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8d621b-8177-4be6-b590-1909e9235353 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:40:50.056240 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-ff8d621b-8177-4be6-b590-1909e9235353 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:40:50.121114 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8d621b-8177-4be6-b590-1909e9235353 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:40:50.126407 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df325f0f-ffa1-4851-bfba-0ecc4caa05f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:40:50.127297 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df325f0f-ffa1-4851-bfba-0ecc4caa05f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:40:50.127583 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df325f0f-ffa1-4851-bfba-0ecc4caa05f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:40:50.127859 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df325f0f-ffa1-4851-bfba-0ecc4caa05f8 admin admin] PATH_INFO: `/v3/users/7dfcd1bd64854895ac45d36c63455181` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:40:50.129382 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df325f0f-ffa1-4851-bfba-0ecc4caa05f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7dfcd1bd64854895ac45d36c63455181)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:40:50.131425 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df325f0f-ffa1-4851-bfba-0ecc4caa05f8 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:40:50.311050 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651687cc-5d44-43c2-9673-3a2bed2e08ef None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:40:50.311623 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651687cc-5d44-43c2-9673-3a2bed2e08ef None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:40:50.311623 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651687cc-5d44-43c2-9673-3a2bed2e08ef None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:40:50.312940 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1171/2347] 10.222.0.33 () {54 vars in 921 bytes} [Fri Jul 15 19:40:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 19:40:50.318913 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc5e245-e803-4031-b380-999780b9ed8f None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:40:50.319200 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc5e245-e803-4031-b380-999780b9ed8f None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:40:50.319480 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc5e245-e803-4031-b380-999780b9ed8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:40:50.338540 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-ccc5e245-e803-4031-b380-999780b9ed8f None None] MFA Rules not processed for user `7673a3b5c0324e07a9535ac70ed7a491`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:40:50.340101 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-ccc5e245-e803-4031-b380-999780b9ed8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:40:50.355540 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1172/2348] 10.222.0.33 () {58 vars in 1018 bytes} [Fri Jul 15 19:40:50 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:40:50.360017 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-df325f0f-ffa1-4851-bfba-0ecc4caa05f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7dfcd1bd64854895ac45d36c63455181'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:40:50.365419 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:40:50.367478 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc5e245-e803-4031-b380-999780b9ed8f None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:40:50.368879 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-ccc5e245-e803-4031-b380-999780b9ed8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:40:50.372720 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-df325f0f-ffa1-4851-bfba-0ecc4caa05f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7dfcd1bd64854895ac45d36c63455181'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:40:50.415067 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:40:50.425183 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc5e245-e803-4031-b380-999780b9ed8f None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:40:50.428925 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a9d2cd-8774-4887-8df4-1ff84c720f01 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04b5b06b0015450a95a28bbcc757cc8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:40:50.429577 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a9d2cd-8774-4887-8df4-1ff84c720f01 service glance] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:40:50.429940 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a9d2cd-8774-4887-8df4-1ff84c720f01 service glance] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:40:50.430179 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a9d2cd-8774-4887-8df4-1ff84c720f01 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:40:50.440638 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a9d2cd-8774-4887-8df4-1ff84c720f01 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:40:50.442212 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a9d2cd-8774-4887-8df4-1ff84c720f01 service glance] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:40:50.453716 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1173/2349] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:40:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 19:40:50.467478 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-df325f0f-ffa1-4851-bfba-0ecc4caa05f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7dfcd1bd64854895ac45d36c63455181'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:40:50.468127 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-df325f0f-ffa1-4851-bfba-0ecc4caa05f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7dfcd1bd64854895ac45d36c63455181'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:40:50.468741 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-df325f0f-ffa1-4851-bfba-0ecc4caa05f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7dfcd1bd64854895ac45d36c63455181'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:40:50.486543 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1177/2350] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jul 15 19:40:50 2022] DELETE /identity/v3/users/7dfcd1bd64854895ac45d36c63455181 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:40:50.491132 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3344c97-2525-41a0-bd77-4cbdb7a2f9c5 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:40:50.491426 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3344c97-2525-41a0-bd77-4cbdb7a2f9c5 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:40:50.491707 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3344c97-2525-41a0-bd77-4cbdb7a2f9c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:40:50.510485 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-e3344c97-2525-41a0-bd77-4cbdb7a2f9c5 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:40:50.511773 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e3344c97-2525-41a0-bd77-4cbdb7a2f9c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:40:50.581446 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1174/2351] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:40:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:40:50.591350 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df325f0f-ffa1-4851-bfba-0ecc4caa05f8 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:40:50.593302 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-df325f0f-ffa1-4851-bfba-0ecc4caa05f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:40:50.651198 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df325f0f-ffa1-4851-bfba-0ecc4caa05f8 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:40:50.654795 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ae0e13-8e49-448b-a728-7d747625a410 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:40:50.655439 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae0e13-8e49-448b-a728-7d747625a410 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:40:50.655719 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae0e13-8e49-448b-a728-7d747625a410 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:40:50.655997 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae0e13-8e49-448b-a728-7d747625a410 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:40:50.662572 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e9ab8e-c728-43a5-9161-d811f9610f4b None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:40:50.662860 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e9ab8e-c728-43a5-9161-d811f9610f4b None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:40:50.663138 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e9ab8e-c728-43a5-9161-d811f9610f4b None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:40:50.664657 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1175/2352] 10.222.0.33 () {54 vars in 921 bytes} [Fri Jul 15 19:40:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:40:50.665783 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ae0e13-8e49-448b-a728-7d747625a410 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:40:50.670332 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ae0e13-8e49-448b-a728-7d747625a410 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:40:50.670448 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5463e5e9-bed9-4276-a6ea-ef92d864ea2a None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:40:50.670448 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5463e5e9-bed9-4276-a6ea-ef92d864ea2a None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:40:50.670448 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5463e5e9-bed9-4276-a6ea-ef92d864ea2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:40:50.676983 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1178/2353] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:40:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:40:50.689240 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-5463e5e9-bed9-4276-a6ea-ef92d864ea2a None None] MFA Rules not processed for user `b2a283d7d3594e37943bda4529d0a775`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:40:50.690475 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-5463e5e9-bed9-4276-a6ea-ef92d864ea2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:40:50.697296 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1176/2354] 10.222.0.33 () {58 vars in 1018 bytes} [Fri Jul 15 19:40:50 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:40:50.712694 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ae0e13-8e49-448b-a728-7d747625a410 service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:40:50.721379 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ae0e13-8e49-448b-a728-7d747625a410 service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:40:50.725043 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91900df6-ac4d-4934-a41f-66013a874203 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:40:50.725684 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91900df6-ac4d-4934-a41f-66013a874203 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:40:50.725997 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91900df6-ac4d-4934-a41f-66013a874203 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:40:50.726275 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91900df6-ac4d-4934-a41f-66013a874203 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:40:50.734894 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91900df6-ac4d-4934-a41f-66013a874203 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:40:50.736327 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91900df6-ac4d-4934-a41f-66013a874203 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:40:50.744821 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1179/2355] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:40:50 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 19:40:50.863361 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36353b6-a4eb-4eb2-a61b-bae9f77631e5 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:40:50.863680 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36353b6-a4eb-4eb2-a61b-bae9f77631e5 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:40:50.863961 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36353b6-a4eb-4eb2-a61b-bae9f77631e5 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:40:50.865379 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1177/2356] 10.222.0.33 () {54 vars in 921 bytes} [Fri Jul 15 19:40:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:40:50.871016 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b13ee35-667b-4f8a-aa43-b602ca201bc9 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:40:50.871308 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b13ee35-667b-4f8a-aa43-b602ca201bc9 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:40:50.871589 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b13ee35-667b-4f8a-aa43-b602ca201bc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:40:50.891612 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-6b13ee35-667b-4f8a-aa43-b602ca201bc9 None None] MFA Rules not processed for user `7673a3b5c0324e07a9535ac70ed7a491`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:40:50.892877 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-6b13ee35-667b-4f8a-aa43-b602ca201bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:40:50.962307 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1180/2357] 10.222.0.33 () {58 vars in 1018 bytes} [Fri Jul 15 19:40:50 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:40:50.972944 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36353b6-a4eb-4eb2-a61b-bae9f77631e5 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:40:50.984879 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36353b6-a4eb-4eb2-a61b-bae9f77631e5 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:40:50.988555 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a9a342-7a41-45b6-b0ce-fe570e1df45a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:40:50.989209 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a9a342-7a41-45b6-b0ce-fe570e1df45a service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:40:50.989493 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a9a342-7a41-45b6-b0ce-fe570e1df45a service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:40:50.989802 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a9a342-7a41-45b6-b0ce-fe570e1df45a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:40:51.001019 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a9a342-7a41-45b6-b0ce-fe570e1df45a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:40:51.002519 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a9a342-7a41-45b6-b0ce-fe570e1df45a service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:40:51.014541 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1178/2358] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:40:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:40:51.077165 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b13ee35-667b-4f8a-aa43-b602ca201bc9 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:40:51.078611 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-6b13ee35-667b-4f8a-aa43-b602ca201bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:40:51.122283 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a9a342-7a41-45b6-b0ce-fe570e1df45a service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:40:51.125202 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-f4a9a342-7a41-45b6-b0ce-fe570e1df45a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:40:51.134855 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:40:51.153245 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a9a342-7a41-45b6-b0ce-fe570e1df45a service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:40:51.157030 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b13ee35-667b-4f8a-aa43-b602ca201bc9 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:40:51.159278 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc7309b-a88f-47ee-9011-a67db4b65d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:40:51.160289 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc7309b-a88f-47ee-9011-a67db4b65d1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:40:51.160647 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc7309b-a88f-47ee-9011-a67db4b65d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:40:51.161429 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc7309b-a88f-47ee-9011-a67db4b65d1d admin admin] PATH_INFO: `/v3/projects/5f679df434534f34bd6ae50dc28ffb0b` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:40:51.162276 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cf4163-daf7-4a48-bbac-0f5c6cb14936 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52a0fce231ba4785a685ac9898d07503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:40:51.162711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc7309b-a88f-47ee-9011-a67db4b65d1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f679df434534f34bd6ae50dc28ffb0b)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:40:51.162946 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cf4163-daf7-4a48-bbac-0f5c6cb14936 service placement] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:40:51.163253 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cf4163-daf7-4a48-bbac-0f5c6cb14936 service placement] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:40:51.163514 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cf4163-daf7-4a48-bbac-0f5c6cb14936 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:40:51.164642 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc7309b-a88f-47ee-9011-a67db4b65d1d admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:40:51.165538 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-23cf4163-daf7-4a48-bbac-0f5c6cb14936 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:40:51.177521 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cf4163-daf7-4a48-bbac-0f5c6cb14936 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:40:51.179059 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cf4163-daf7-4a48-bbac-0f5c6cb14936 service placement] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:40:51.189142 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1181/2359] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:40:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:40:51.292350 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ecc7309b-a88f-47ee-9011-a67db4b65d1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f679df434534f34bd6ae50dc28ffb0b', 'request_id': 'req-ecc7309b-a88f-47ee-9011-a67db4b65d1d'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:40:51.404334 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ecc7309b-a88f-47ee-9011-a67db4b65d1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f679df434534f34bd6ae50dc28ffb0b', 'request_id': 'req-ecc7309b-a88f-47ee-9011-a67db4b65d1d'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:40:51.411514 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ecc7309b-a88f-47ee-9011-a67db4b65d1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f679df434534f34bd6ae50dc28ffb0b', 'request_id': 'req-ecc7309b-a88f-47ee-9011-a67db4b65d1d'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:40:51.427787 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ecc7309b-a88f-47ee-9011-a67db4b65d1d admin admin] The token cache is being invalidate because project 5f679df434534f34bd6ae50dc28ffb0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:40:51.428055 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ecc7309b-a88f-47ee-9011-a67db4b65d1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:40:51.428704 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ecc7309b-a88f-47ee-9011-a67db4b65d1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:40:51.430545 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1179/2360] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jul 15 19:40:51 2022] DELETE /identity/v3/projects/5f679df434534f34bd6ae50dc28ffb0b => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:40:51.436932 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c03e70-d795-4e0b-ab0c-0a4095a65fe8 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:40:51.436932 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c03e70-d795-4e0b-ab0c-0a4095a65fe8 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:40:51.436932 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c03e70-d795-4e0b-ab0c-0a4095a65fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:40:51.464888 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-a5c03e70-d795-4e0b-ab0c-0a4095a65fe8 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:40:51.466141 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-a5c03e70-d795-4e0b-ab0c-0a4095a65fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:40:51.537550 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1182/2361] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:40:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:40:51.598567 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eb48c0-f082-4967-9a67-2d142818e270 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:40:51.598988 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eb48c0-f082-4967-9a67-2d142818e270 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:40:51.599347 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eb48c0-f082-4967-9a67-2d142818e270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:40:51.626493 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-95eb48c0-f082-4967-9a67-2d142818e270 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:40:51.627806 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-95eb48c0-f082-4967-9a67-2d142818e270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:40:51.637461 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1180/2362] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:40:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:40:51.641657 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c03e70-d795-4e0b-ab0c-0a4095a65fe8 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:40:51.651695 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c03e70-d795-4e0b-ab0c-0a4095a65fe8 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:40:51.657129 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873c3935-ab4d-4ae7-938e-d873f855bd92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:40:51.657821 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873c3935-ab4d-4ae7-938e-d873f855bd92 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:40:51.659491 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873c3935-ab4d-4ae7-938e-d873f855bd92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:40:51.659491 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873c3935-ab4d-4ae7-938e-d873f855bd92 admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:40:51.659491 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873c3935-ab4d-4ae7-938e-d873f855bd92 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:40:51.659491 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873c3935-ab4d-4ae7-938e-d873f855bd92 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:40:51.661245 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873c3935-ab4d-4ae7-938e-d873f855bd92 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:40:51.684883 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1183/2363] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jul 15 19:40:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:40:51.685563 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eb48c0-f082-4967-9a67-2d142818e270 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:40:51.692761 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eb48c0-f082-4967-9a67-2d142818e270 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:40:51.697572 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d17ca70-fb6f-4545-a95f-d80078dd2061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:40:51.698246 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d17ca70-fb6f-4545-a95f-d80078dd2061 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:40:51.698527 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d17ca70-fb6f-4545-a95f-d80078dd2061 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:40:51.698859 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d17ca70-fb6f-4545-a95f-d80078dd2061 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:40:51.699550 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d17ca70-fb6f-4545-a95f-d80078dd2061 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:40:51.702199 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d17ca70-fb6f-4545-a95f-d80078dd2061 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:40:51.816966 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1181/2364] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jul 15 19:40:51 2022] POST /identity/v3/projects => generated 399 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:40:51.818480 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873c3935-ab4d-4ae7-938e-d873f855bd92 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:40:51.828508 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873c3935-ab4d-4ae7-938e-d873f855bd92 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:40:51.889006 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddb30f6-932b-46c2-89c8-bdc92a0ed201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:40:51.889680 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddb30f6-932b-46c2-89c8-bdc92a0ed201 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:40:51.889992 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddb30f6-932b-46c2-89c8-bdc92a0ed201 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:40:51.890267 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddb30f6-932b-46c2-89c8-bdc92a0ed201 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:40:51.890951 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddb30f6-932b-46c2-89c8-bdc92a0ed201 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:40:51.892895 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddb30f6-932b-46c2-89c8-bdc92a0ed201 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:40:51.966556 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1184/2365] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jul 15 19:40:51 2022] POST /identity/v3/users => generated 300 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:40:51.970858 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d17ca70-fb6f-4545-a95f-d80078dd2061 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:40:51.981584 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d17ca70-fb6f-4545-a95f-d80078dd2061 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:40:51.986952 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d838a9-ec19-4846-a044-988417e3cdad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:40:52.009675 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d838a9-ec19-4846-a044-988417e3cdad admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:40:52.009675 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d838a9-ec19-4846-a044-988417e3cdad admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:40:52.009675 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d838a9-ec19-4846-a044-988417e3cdad admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:40:52.009675 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d838a9-ec19-4846-a044-988417e3cdad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:40:52.009675 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d838a9-ec19-4846-a044-988417e3cdad admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:40:52.009675 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1182/2366] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:40:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:40:52.012512 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddb30f6-932b-46c2-89c8-bdc92a0ed201 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:40:52.023227 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddb30f6-932b-46c2-89c8-bdc92a0ed201 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:40:52.027899 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6f932e-39d9-4c5d-86af-888cfd8b7758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:40:52.028546 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f932e-39d9-4c5d-86af-888cfd8b7758 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:40:52.029536 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f932e-39d9-4c5d-86af-888cfd8b7758 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:40:52.030069 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f932e-39d9-4c5d-86af-888cfd8b7758 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:40:52.030783 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6f932e-39d9-4c5d-86af-888cfd8b7758 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:40:52.032375 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6f932e-39d9-4c5d-86af-888cfd8b7758 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:40:52.042242 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1185/2367] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:40:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:40:52.046362 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d838a9-ec19-4846-a044-988417e3cdad admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:40:52.056338 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d838a9-ec19-4846-a044-988417e3cdad admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:40:52.061336 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b734fc25-640f-4cb0-bb24-2d4f1d93bb9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:40:52.062261 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b734fc25-640f-4cb0-bb24-2d4f1d93bb9b admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:40:52.063093 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b734fc25-640f-4cb0-bb24-2d4f1d93bb9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:40:52.063093 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b734fc25-640f-4cb0-bb24-2d4f1d93bb9b admin admin] PATH_INFO: `/v3/projects/610d94f411414c48bc0839f0e223cbc7/users/ce253da9ebe844f288a1ac455a343b84/roles/28763c6e70094116b3a4b2895895fa58` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:40:52.071008 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:40:52.078371 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b734fc25-640f-4cb0-bb24-2d4f1d93bb9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=610d94f411414c48bc0839f0e223cbc7, user_id=ce253da9ebe844f288a1ac455a343b84, role_id=28763c6e70094116b3a4b2895895fa58)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:40:52.081016 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b734fc25-640f-4cb0-bb24-2d4f1d93bb9b admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:40:52.189984 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1183/2368] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:40:52 2022] PUT /identity/v3/projects/610d94f411414c48bc0839f0e223cbc7/users/ce253da9ebe844f288a1ac455a343b84/roles/28763c6e70094116b3a4b2895895fa58 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:40:52.200030 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c98eb7-ecdb-41ad-a608-021a66239e37 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:40:52.200501 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c98eb7-ecdb-41ad-a608-021a66239e37 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:40:52.200620 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c98eb7-ecdb-41ad-a608-021a66239e37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:40:52.228484 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-f1c98eb7-ecdb-41ad-a608-021a66239e37 None None] MFA Rules not processed for user `ce253da9ebe844f288a1ac455a343b84`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:40:52.229675 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-f1c98eb7-ecdb-41ad-a608-021a66239e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:40:52.272562 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:40:52.299796 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1186/2369] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:40:52 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:40:52.311321 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b734fc25-640f-4cb0-bb24-2d4f1d93bb9b admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:40:52.313101 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b734fc25-640f-4cb0-bb24-2d4f1d93bb9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:40:52.372772 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b734fc25-640f-4cb0-bb24-2d4f1d93bb9b admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:40:52.376470 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696b6eb5-248b-4d8e-baf8-c16e05424bc9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86c2df3d378490ca7ea497a73c19d40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:40:52.377115 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b6eb5-248b-4d8e-baf8-c16e05424bc9 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:40:52.377392 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b6eb5-248b-4d8e-baf8-c16e05424bc9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:40:52.377663 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b6eb5-248b-4d8e-baf8-c16e05424bc9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:40:52.386630 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696b6eb5-248b-4d8e-baf8-c16e05424bc9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:40:52.388046 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696b6eb5-248b-4d8e-baf8-c16e05424bc9 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:40:52.397147 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1184/2370] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:40:52 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:40:55.341734 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a8fe7f-614e-48b2-b592-fb568b072481 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:40:55.342267 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a8fe7f-614e-48b2-b592-fb568b072481 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:40:55.342561 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a8fe7f-614e-48b2-b592-fb568b072481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:40:55.360685 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-f6a8fe7f-614e-48b2-b592-fb568b072481 None None] MFA Rules not processed for user `3d893c14620b4a129842062f14a68a78`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:40:55.361920 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-f6a8fe7f-614e-48b2-b592-fb568b072481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:40:55.419172 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1187/2371] 10.222.0.33 () {58 vars in 1009 bytes} [Fri Jul 15 19:40:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:40:55.431906 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696b6eb5-248b-4d8e-baf8-c16e05424bc9 service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:40:55.433384 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-696b6eb5-248b-4d8e-baf8-c16e05424bc9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:40:55.502687 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696b6eb5-248b-4d8e-baf8-c16e05424bc9 service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:40:55.508156 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18500cf0-985e-4afa-93ad-d7fc16ee5a03 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef76f7fe39a3499aaeebf85c176804f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:40:55.508806 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18500cf0-985e-4afa-93ad-d7fc16ee5a03 service swift] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:40:55.509090 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18500cf0-985e-4afa-93ad-d7fc16ee5a03 service swift] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:40:55.509371 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18500cf0-985e-4afa-93ad-d7fc16ee5a03 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:40:55.517643 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18500cf0-985e-4afa-93ad-d7fc16ee5a03 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:40:55.519263 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18500cf0-985e-4afa-93ad-d7fc16ee5a03 service swift] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:40:55.527651 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1185/2372] 10.222.0.33 () {60 vars in 1384 bytes} [Fri Jul 15 19:40:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 19:41:05.411808 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=35 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:41:05.524166 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=63,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:41:05.785568 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a8fe7f-614e-48b2-b592-fb568b072481 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:41:05.853284 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a8fe7f-614e-48b2-b592-fb568b072481 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:41:05.858258 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14988d7-1f8b-4684-8912-938778077f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:41:05.859143 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14988d7-1f8b-4684-8912-938778077f13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:41:05.859423 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14988d7-1f8b-4684-8912-938778077f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:41:05.859698 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14988d7-1f8b-4684-8912-938778077f13 admin admin] PATH_INFO: `/v3/users/ce253da9ebe844f288a1ac455a343b84` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:41:05.860919 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14988d7-1f8b-4684-8912-938778077f13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce253da9ebe844f288a1ac455a343b84)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:41:05.862884 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14988d7-1f8b-4684-8912-938778077f13 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:41:06.109283 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-f14988d7-1f8b-4684-8912-938778077f13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce253da9ebe844f288a1ac455a343b84'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:41:06.121053 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-f14988d7-1f8b-4684-8912-938778077f13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce253da9ebe844f288a1ac455a343b84'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:41:06.205533 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-f14988d7-1f8b-4684-8912-938778077f13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce253da9ebe844f288a1ac455a343b84'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:41:06.206205 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-f14988d7-1f8b-4684-8912-938778077f13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce253da9ebe844f288a1ac455a343b84'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:41:06.206816 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-f14988d7-1f8b-4684-8912-938778077f13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce253da9ebe844f288a1ac455a343b84'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:41:06.220750 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1188/2373] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jul 15 19:41:05 2022] DELETE /identity/v3/users/ce253da9ebe844f288a1ac455a343b84 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:41:06.225376 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af93d87-8131-4399-80f1-d1fc60aca3ed None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:41:06.225659 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af93d87-8131-4399-80f1-d1fc60aca3ed None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:41:06.231610 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af93d87-8131-4399-80f1-d1fc60aca3ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:41:06.243394 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-0af93d87-8131-4399-80f1-d1fc60aca3ed None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:41:06.244664 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0af93d87-8131-4399-80f1-d1fc60aca3ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:41:06.320106 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1186/2374] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:41:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:41:06.330754 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14988d7-1f8b-4684-8912-938778077f13 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:41:06.332190 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-f14988d7-1f8b-4684-8912-938778077f13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:41:06.391605 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14988d7-1f8b-4684-8912-938778077f13 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:41:06.395667 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ebacdd-22e5-40e3-a551-04b4950054a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:41:06.396338 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ebacdd-22e5-40e3-a551-04b4950054a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:41:06.396627 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ebacdd-22e5-40e3-a551-04b4950054a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:41:06.396907 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ebacdd-22e5-40e3-a551-04b4950054a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:41:06.408093 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ebacdd-22e5-40e3-a551-04b4950054a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:41:06.409856 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ebacdd-22e5-40e3-a551-04b4950054a5 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:41:06.420592 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1189/2375] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:41:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:41:07.070951 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af93d87-8131-4399-80f1-d1fc60aca3ed None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:41:07.072892 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0af93d87-8131-4399-80f1-d1fc60aca3ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:41:07.086460 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af93d87-8131-4399-80f1-d1fc60aca3ed None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:41:07.091521 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ac4d19-4846-4b20-9e70-e3eaf0da073b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:41:07.092421 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ac4d19-4846-4b20-9e70-e3eaf0da073b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:41:07.092695 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ac4d19-4846-4b20-9e70-e3eaf0da073b admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:41:07.092976 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ac4d19-4846-4b20-9e70-e3eaf0da073b admin admin] PATH_INFO: `/v3/projects/610d94f411414c48bc0839f0e223cbc7` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:41:07.094238 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ac4d19-4846-4b20-9e70-e3eaf0da073b admin admin] RBAC: Authorizing `identity:delete_project(project_id=610d94f411414c48bc0839f0e223cbc7)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:41:07.096157 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ac4d19-4846-4b20-9e70-e3eaf0da073b admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:41:07.222724 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-a3ac4d19-4846-4b20-9e70-e3eaf0da073b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '610d94f411414c48bc0839f0e223cbc7', 'request_id': 'req-a3ac4d19-4846-4b20-9e70-e3eaf0da073b'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:41:07.302927 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-a3ac4d19-4846-4b20-9e70-e3eaf0da073b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '610d94f411414c48bc0839f0e223cbc7', 'request_id': 'req-a3ac4d19-4846-4b20-9e70-e3eaf0da073b'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:41:07.309303 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-a3ac4d19-4846-4b20-9e70-e3eaf0da073b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '610d94f411414c48bc0839f0e223cbc7', 'request_id': 'req-a3ac4d19-4846-4b20-9e70-e3eaf0da073b'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:41:07.321499 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-a3ac4d19-4846-4b20-9e70-e3eaf0da073b admin admin] The token cache is being invalidate because project 610d94f411414c48bc0839f0e223cbc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:41:07.321702 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-a3ac4d19-4846-4b20-9e70-e3eaf0da073b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:41:07.322435 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-a3ac4d19-4846-4b20-9e70-e3eaf0da073b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:41:07.324316 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1187/2376] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jul 15 19:41:07 2022] DELETE /identity/v3/projects/610d94f411414c48bc0839f0e223cbc7 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:41:16.415910 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:41:17.318213 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:41:24.502015 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ebacdd-22e5-40e3-a551-04b4950054a5 service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:41:24.503724 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-f4ebacdd-22e5-40e3-a551-04b4950054a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:41:24.515633 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ebacdd-22e5-40e3-a551-04b4950054a5 service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:41:24.563498 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2396f0-2a67-4f31-9082-6d07c6afdfe8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:41:24.564155 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2396f0-2a67-4f31-9082-6d07c6afdfe8 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:41:24.564440 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2396f0-2a67-4f31-9082-6d07c6afdfe8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:41:24.564750 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2396f0-2a67-4f31-9082-6d07c6afdfe8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:41:24.566559 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-4a2396f0-2a67-4f31-9082-6d07c6afdfe8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:41:24.576446 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2396f0-2a67-4f31-9082-6d07c6afdfe8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:41:24.578512 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2396f0-2a67-4f31-9082-6d07c6afdfe8 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:41:24.586513 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1190/2377] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:41:24 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:41:26.971949 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887bc61-4c96-496d-9b94-90f566c67ace None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:41:26.972435 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887bc61-4c96-496d-9b94-90f566c67ace None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:41:26.972535 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887bc61-4c96-496d-9b94-90f566c67ace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:41:26.990387 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-a887bc61-4c96-496d-9b94-90f566c67ace None None] MFA Rules not processed for user `3d893c14620b4a129842062f14a68a78`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:41:26.991592 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-a887bc61-4c96-496d-9b94-90f566c67ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:41:27.047868 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1188/2378] 10.222.0.33 () {58 vars in 967 bytes} [Fri Jul 15 19:41:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:41:27.060550 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2396f0-2a67-4f31-9082-6d07c6afdfe8 service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:41:27.061906 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-4a2396f0-2a67-4f31-9082-6d07c6afdfe8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:41:27.134082 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2396f0-2a67-4f31-9082-6d07c6afdfe8 service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:41:27.139452 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72893894-f380-450d-921e-7bba8162a260 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef76f7fe39a3499aaeebf85c176804f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:41:27.140099 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72893894-f380-450d-921e-7bba8162a260 service swift] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:41:27.140375 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72893894-f380-450d-921e-7bba8162a260 service swift] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:41:27.140645 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72893894-f380-450d-921e-7bba8162a260 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:41:27.149268 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72893894-f380-450d-921e-7bba8162a260 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:41:27.150858 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72893894-f380-450d-921e-7bba8162a260 service swift] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:41:27.159376 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1191/2379] 10.222.0.33 () {60 vars in 1384 bytes} [Fri Jul 15 19:41:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 19:41:37.041128 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=25 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:41:37.155485 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=58 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:41:44.400495 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a887bc61-4c96-496d-9b94-90f566c67ace None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:41:44.401950 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-a887bc61-4c96-496d-9b94-90f566c67ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:41:44.474812 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a887bc61-4c96-496d-9b94-90f566c67ace None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:41:44.480280 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d04052-bc9e-407a-a7f1-155516e03db8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7673a3b5c0324e07a9535ac70ed7a491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:41:44.480932 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d04052-bc9e-407a-a7f1-155516e03db8 service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:41:44.481217 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d04052-bc9e-407a-a7f1-155516e03db8 service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:41:44.481499 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d04052-bc9e-407a-a7f1-155516e03db8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:41:44.483305 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-69d04052-bc9e-407a-a7f1-155516e03db8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:41:44.548812 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d04052-bc9e-407a-a7f1-155516e03db8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:41:44.551060 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d04052-bc9e-407a-a7f1-155516e03db8 service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:41:44.560038 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1189/2380] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:41:44 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:41:54.555853 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=56 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:42:09.809247 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72fb089-cbb4-48c6-9f47-cb7ae74a44c7 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:42:09.809733 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72fb089-cbb4-48c6-9f47-cb7ae74a44c7 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:42:09.809878 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72fb089-cbb4-48c6-9f47-cb7ae74a44c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:42:09.826560 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-b72fb089-cbb4-48c6-9f47-cb7ae74a44c7 None None] MFA Rules not processed for user `8a654f77867341a1adf8db7f36ddc653`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:42:09.827758 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-b72fb089-cbb4-48c6-9f47-cb7ae74a44c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:42:09.834722 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1192/2381] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:42:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:42:16.366186 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d04052-bc9e-407a-a7f1-155516e03db8 service nova] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:42:16.368255 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-69d04052-bc9e-407a-a7f1-155516e03db8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:42:16.383427 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d04052-bc9e-407a-a7f1-155516e03db8 service nova] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:42:16.391233 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5468ce95-5e39-4ba0-abaf-5f3bed1e4396 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7673a3b5c0324e07a9535ac70ed7a491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:42:16.391909 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5468ce95-5e39-4ba0-abaf-5f3bed1e4396 service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:42:16.392374 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5468ce95-5e39-4ba0-abaf-5f3bed1e4396 service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:42:16.392671 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5468ce95-5e39-4ba0-abaf-5f3bed1e4396 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:42:16.394664 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-5468ce95-5e39-4ba0-abaf-5f3bed1e4396 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:42:16.407418 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5468ce95-5e39-4ba0-abaf-5f3bed1e4396 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:42:16.409020 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5468ce95-5e39-4ba0-abaf-5f3bed1e4396 service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:42:16.420854 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1190/2382] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:42:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:42:19.821836 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:42:26.416008 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=6 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:42:42.832188 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72fb089-cbb4-48c6-9f47-cb7ae74a44c7 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:42:42.848260 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72fb089-cbb4-48c6-9f47-cb7ae74a44c7 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:42:42.853741 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c94f87c-8c72-4e79-8f2a-c5ad1e740abc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7673a3b5c0324e07a9535ac70ed7a491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:42:42.854442 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c94f87c-8c72-4e79-8f2a-c5ad1e740abc service nova] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:42:42.854725 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c94f87c-8c72-4e79-8f2a-c5ad1e740abc service nova] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:42:42.854999 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c94f87c-8c72-4e79-8f2a-c5ad1e740abc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:42:42.856824 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-5c94f87c-8c72-4e79-8f2a-c5ad1e740abc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:42:42.871256 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c94f87c-8c72-4e79-8f2a-c5ad1e740abc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:42:42.872858 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c94f87c-8c72-4e79-8f2a-c5ad1e740abc service nova] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:42:42.885895 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1193/2383] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:42:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:42:48.461477 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ee0eae-a31d-49b7-924a-bc0291786a29 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:42:48.462018 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ee0eae-a31d-49b7-924a-bc0291786a29 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:42:48.462135 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ee0eae-a31d-49b7-924a-bc0291786a29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:42:48.479252 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-c3ee0eae-a31d-49b7-924a-bc0291786a29 None None] MFA Rules not processed for user `6193fdc75def489e88eda70e78ec6f46`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:42:48.480499 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-c3ee0eae-a31d-49b7-924a-bc0291786a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:42:48.536856 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1191/2384] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:42:48 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:42:48.846541 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c94f87c-8c72-4e79-8f2a-c5ad1e740abc service nova] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:42:48.848039 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-5c94f87c-8c72-4e79-8f2a-c5ad1e740abc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:42:48.859803 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c94f87c-8c72-4e79-8f2a-c5ad1e740abc service nova] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:42:48.863439 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49672e7a-87d3-4e96-aaf8-5d39dc240355 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:42:48.864103 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49672e7a-87d3-4e96-aaf8-5d39dc240355 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:42:48.864386 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49672e7a-87d3-4e96-aaf8-5d39dc240355 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:42:48.864661 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49672e7a-87d3-4e96-aaf8-5d39dc240355 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:42:48.866423 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-49672e7a-87d3-4e96-aaf8-5d39dc240355 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:42:48.875896 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49672e7a-87d3-4e96-aaf8-5d39dc240355 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:42:48.877308 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49672e7a-87d3-4e96-aaf8-5d39dc240355 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:42:48.885181 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1194/2385] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:42:48 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:42:58.529421 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=25 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:42:58.881675 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=12 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:43:00.335970 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ee0eae-a31d-49b7-924a-bc0291786a29 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:43:00.348680 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ee0eae-a31d-49b7-924a-bc0291786a29 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:43:00.354151 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0672e877-5985-4276-bdb0-dc05d56e65dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7673a3b5c0324e07a9535ac70ed7a491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:43:00.354800 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0672e877-5985-4276-bdb0-dc05d56e65dc service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:43:00.355083 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0672e877-5985-4276-bdb0-dc05d56e65dc service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:43:00.355363 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0672e877-5985-4276-bdb0-dc05d56e65dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:43:00.357155 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0672e877-5985-4276-bdb0-dc05d56e65dc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:43:00.369869 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0672e877-5985-4276-bdb0-dc05d56e65dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:43:00.371464 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0672e877-5985-4276-bdb0-dc05d56e65dc service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:43:00.382969 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1192/2386] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:43:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:43:10.377785 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=6 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:43:19.439714 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae1f89-d13e-47dc-8190-f1a0a1a1c45c None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:43:19.440176 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae1f89-d13e-47dc-8190-f1a0a1a1c45c None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:43:19.440308 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae1f89-d13e-47dc-8190-f1a0a1a1c45c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:43:19.458368 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-85ae1f89-d13e-47dc-8190-f1a0a1a1c45c None None] MFA Rules not processed for user `41d65200f05f4096823ecc0229b1ce3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:43:19.459598 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-85ae1f89-d13e-47dc-8190-f1a0a1a1c45c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:43:19.514758 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1195/2387] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:43:19 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:43:19.824617 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0672e877-5985-4276-bdb0-dc05d56e65dc service nova] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:43:19.826184 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0672e877-5985-4276-bdb0-dc05d56e65dc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:43:19.838171 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0672e877-5985-4276-bdb0-dc05d56e65dc service nova] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:43:19.841737 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a810d9-e41f-485b-adba-083914fec9df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:43:19.842478 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a810d9-e41f-485b-adba-083914fec9df service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:43:19.842762 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a810d9-e41f-485b-adba-083914fec9df service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:43:19.843034 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a810d9-e41f-485b-adba-083914fec9df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:43:19.844766 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-86a810d9-e41f-485b-adba-083914fec9df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:43:19.855016 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a810d9-e41f-485b-adba-083914fec9df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:43:19.856418 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a810d9-e41f-485b-adba-083914fec9df service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:43:19.864263 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1193/2388] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:43:19 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:43:29.507504 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=25 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:43:29.860785 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=6 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:43:47.712320 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68851511-3af3-4627-b1c1-d846565be7ef None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:43:47.712812 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68851511-3af3-4627-b1c1-d846565be7ef None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:43:47.712905 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68851511-3af3-4627-b1c1-d846565be7ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:43:47.742245 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-68851511-3af3-4627-b1c1-d846565be7ef None None] MFA Rules not processed for user `eb2902b42e3449ba95b73ad5f85fcfdc`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:43:47.743381 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-68851511-3af3-4627-b1c1-d846565be7ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:43:47.814907 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1196/2389] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:43:47 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:43:48.212853 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a810d9-e41f-485b-adba-083914fec9df service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:43:48.244506 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a810d9-e41f-485b-adba-083914fec9df service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:43:48.248236 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e4d88f-0c9d-4467-8c0d-47c9f5e8d9d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:43:48.248918 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e4d88f-0c9d-4467-8c0d-47c9f5e8d9d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:43:48.249453 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e4d88f-0c9d-4467-8c0d-47c9f5e8d9d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:43:48.249731 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e4d88f-0c9d-4467-8c0d-47c9f5e8d9d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:43:48.251558 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-c2e4d88f-0c9d-4467-8c0d-47c9f5e8d9d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:43:48.352667 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e4d88f-0c9d-4467-8c0d-47c9f5e8d9d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:43:48.354197 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e4d88f-0c9d-4467-8c0d-47c9f5e8d9d2 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:43:48.373931 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1194/2390] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:43:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:43:52.623650 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68851511-3af3-4627-b1c1-d846565be7ef None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:43:52.625557 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-68851511-3af3-4627-b1c1-d846565be7ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:43:52.777379 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68851511-3af3-4627-b1c1-d846565be7ef None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:43:52.777488 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e4d88f-0c9d-4467-8c0d-47c9f5e8d9d2 service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:43:52.778759 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-c2e4d88f-0c9d-4467-8c0d-47c9f5e8d9d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:43:52.781861 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bb9850-6bb6-4807-aeb0-e3c21dc3a6a0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52a0fce231ba4785a685ac9898d07503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:43:52.783025 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bb9850-6bb6-4807-aeb0-e3c21dc3a6a0 service placement] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:43:52.783365 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bb9850-6bb6-4807-aeb0-e3c21dc3a6a0 service placement] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:43:52.783644 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bb9850-6bb6-4807-aeb0-e3c21dc3a6a0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:43:52.786125 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-a3bb9850-6bb6-4807-aeb0-e3c21dc3a6a0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:43:52.795435 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e4d88f-0c9d-4467-8c0d-47c9f5e8d9d2 service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:43:52.802540 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ece950-fdf3-4690-a1a5-688a5bfd6f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:43:52.808509 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ece950-fdf3-4690-a1a5-688a5bfd6f1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:43:52.808509 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ece950-fdf3-4690-a1a5-688a5bfd6f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:43:52.808509 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ece950-fdf3-4690-a1a5-688a5bfd6f1b admin admin] PATH_INFO: `/v3/users/8a654f77867341a1adf8db7f36ddc653` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:43:52.808509 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ece950-fdf3-4690-a1a5-688a5bfd6f1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a654f77867341a1adf8db7f36ddc653)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:43:52.809384 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ece950-fdf3-4690-a1a5-688a5bfd6f1b admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:43:52.815708 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bb9850-6bb6-4807-aeb0-e3c21dc3a6a0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:43:52.821065 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bb9850-6bb6-4807-aeb0-e3c21dc3a6a0 service placement] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:43:52.835851 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1197/2391] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:43:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:43:53.088612 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-f6ece950-fdf3-4690-a1a5-688a5bfd6f1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a654f77867341a1adf8db7f36ddc653'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:43:53.104974 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-f6ece950-fdf3-4690-a1a5-688a5bfd6f1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a654f77867341a1adf8db7f36ddc653'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:43:53.201383 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-f6ece950-fdf3-4690-a1a5-688a5bfd6f1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a654f77867341a1adf8db7f36ddc653'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:43:53.202586 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-f6ece950-fdf3-4690-a1a5-688a5bfd6f1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a654f77867341a1adf8db7f36ddc653'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:43:53.211857 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-f6ece950-fdf3-4690-a1a5-688a5bfd6f1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a654f77867341a1adf8db7f36ddc653'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:43:53.218775 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1195/2392] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jul 15 19:43:52 2022] DELETE /identity/v3/users/8a654f77867341a1adf8db7f36ddc653 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:43:53.873214 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bb9850-6bb6-4807-aeb0-e3c21dc3a6a0 service placement] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:43:53.885912 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bb9850-6bb6-4807-aeb0-e3c21dc3a6a0 service placement] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:43:53.934267 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:43:53.946520 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a130a6f-c6bf-49ce-a001-c5a573b38d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:43:53.947387 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a130a6f-c6bf-49ce-a001-c5a573b38d64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:43:53.947668 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a130a6f-c6bf-49ce-a001-c5a573b38d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:43:53.947965 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a130a6f-c6bf-49ce-a001-c5a573b38d64 admin admin] PATH_INFO: `/v3/projects/6793b8b5e7e24faf9b5e3feb466adc08` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:43:53.949155 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a130a6f-c6bf-49ce-a001-c5a573b38d64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6793b8b5e7e24faf9b5e3feb466adc08)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:43:53.951103 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a130a6f-c6bf-49ce-a001-c5a573b38d64 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:43:54.219326 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-8a130a6f-c6bf-49ce-a001-c5a573b38d64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6793b8b5e7e24faf9b5e3feb466adc08', 'request_id': 'req-8a130a6f-c6bf-49ce-a001-c5a573b38d64'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:43:54.423958 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-8a130a6f-c6bf-49ce-a001-c5a573b38d64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6793b8b5e7e24faf9b5e3feb466adc08', 'request_id': 'req-8a130a6f-c6bf-49ce-a001-c5a573b38d64'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:43:54.431474 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-8a130a6f-c6bf-49ce-a001-c5a573b38d64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6793b8b5e7e24faf9b5e3feb466adc08', 'request_id': 'req-8a130a6f-c6bf-49ce-a001-c5a573b38d64'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:43:54.441842 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-8a130a6f-c6bf-49ce-a001-c5a573b38d64 admin admin] The token cache is being invalidate because project 6793b8b5e7e24faf9b5e3feb466adc08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:43:54.442131 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-8a130a6f-c6bf-49ce-a001-c5a573b38d64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:43:54.442758 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-8a130a6f-c6bf-49ce-a001-c5a573b38d64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:43:54.444393 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1198/2393] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jul 15 19:43:53 2022] DELETE /identity/v3/projects/6793b8b5e7e24faf9b5e3feb466adc08 => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:43:54.449986 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cdcff6-80f1-4855-950a-578b43925e56 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:43:54.450477 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cdcff6-80f1-4855-950a-578b43925e56 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:43:54.450592 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cdcff6-80f1-4855-950a-578b43925e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:43:54.454970 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:43:54.510154 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-91cdcff6-80f1-4855-950a-578b43925e56 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:43:54.511372 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-91cdcff6-80f1-4855-950a-578b43925e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:43:54.571495 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1196/2394] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:43:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:43:54.579717 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c548b16-bd5f-4007-9f9e-2f74c2a3b72a None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:43:54.580083 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c548b16-bd5f-4007-9f9e-2f74c2a3b72a None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:43:54.580548 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c548b16-bd5f-4007-9f9e-2f74c2a3b72a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:43:54.598499 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-4c548b16-bd5f-4007-9f9e-2f74c2a3b72a None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:43:54.600653 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-4c548b16-bd5f-4007-9f9e-2f74c2a3b72a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:43:54.607937 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1199/2395] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:43:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:43:54.612253 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cdcff6-80f1-4855-950a-578b43925e56 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:43:54.623027 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cdcff6-80f1-4855-950a-578b43925e56 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:43:54.634809 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db27817e-efc6-4c21-85a6-68e98a2a38b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:43:54.635487 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db27817e-efc6-4c21-85a6-68e98a2a38b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:43:54.635774 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db27817e-efc6-4c21-85a6-68e98a2a38b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:43:54.636053 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db27817e-efc6-4c21-85a6-68e98a2a38b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:43:54.636792 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db27817e-efc6-4c21-85a6-68e98a2a38b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:43:54.637101 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db27817e-efc6-4c21-85a6-68e98a2a38b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:43:54.638743 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db27817e-efc6-4c21-85a6-68e98a2a38b0 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:43:54.650977 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1197/2396] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jul 15 19:43:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:43:54.655121 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c548b16-bd5f-4007-9f9e-2f74c2a3b72a None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:43:54.664300 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c548b16-bd5f-4007-9f9e-2f74c2a3b72a None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:43:54.668948 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52c1392-7171-4ede-9e6a-10ebb3130d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:43:54.669625 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52c1392-7171-4ede-9e6a-10ebb3130d00 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:43:54.669943 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52c1392-7171-4ede-9e6a-10ebb3130d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:43:54.670220 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52c1392-7171-4ede-9e6a-10ebb3130d00 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:43:54.670921 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52c1392-7171-4ede-9e6a-10ebb3130d00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:43:54.672821 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52c1392-7171-4ede-9e6a-10ebb3130d00 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:43:54.802916 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1200/2397] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jul 15 19:43:54 2022] POST /identity/v3/projects => generated 381 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:43:54.807656 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db27817e-efc6-4c21-85a6-68e98a2a38b0 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:43:54.819000 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db27817e-efc6-4c21-85a6-68e98a2a38b0 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:43:54.875782 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c1eae0-d594-40d1-8caa-072c0eb2998d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:43:54.876449 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c1eae0-d594-40d1-8caa-072c0eb2998d admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:43:54.876728 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c1eae0-d594-40d1-8caa-072c0eb2998d admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:43:54.877002 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c1eae0-d594-40d1-8caa-072c0eb2998d admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:43:54.877691 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c1eae0-d594-40d1-8caa-072c0eb2998d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:43:54.879625 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c1eae0-d594-40d1-8caa-072c0eb2998d admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:43:54.966839 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1198/2398] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jul 15 19:43:54 2022] POST /identity/v3/users => generated 291 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:43:54.970947 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52c1392-7171-4ede-9e6a-10ebb3130d00 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:43:54.982045 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52c1392-7171-4ede-9e6a-10ebb3130d00 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:43:54.986733 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e4d328-9e25-4400-bac6-d7ff12fa7a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:43:54.987396 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e4d328-9e25-4400-bac6-d7ff12fa7a8f admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:43:54.987738 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e4d328-9e25-4400-bac6-d7ff12fa7a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:43:54.988016 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e4d328-9e25-4400-bac6-d7ff12fa7a8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:43:54.988781 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e4d328-9e25-4400-bac6-d7ff12fa7a8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:43:54.990403 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e4d328-9e25-4400-bac6-d7ff12fa7a8f admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:43:55.001314 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1201/2399] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:43:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:43:55.005228 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c1eae0-d594-40d1-8caa-072c0eb2998d admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:43:55.015453 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c1eae0-d594-40d1-8caa-072c0eb2998d admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:43:55.020224 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25c1295-0e79-4191-b33b-25734f6b0492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:43:55.020883 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25c1295-0e79-4191-b33b-25734f6b0492 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:43:55.021165 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25c1295-0e79-4191-b33b-25734f6b0492 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:43:55.021443 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25c1295-0e79-4191-b33b-25734f6b0492 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:43:55.022206 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25c1295-0e79-4191-b33b-25734f6b0492 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:43:55.023796 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25c1295-0e79-4191-b33b-25734f6b0492 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:43:55.034205 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1199/2400] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:43:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:43:55.039070 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e4d328-9e25-4400-bac6-d7ff12fa7a8f admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:43:55.048905 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e4d328-9e25-4400-bac6-d7ff12fa7a8f admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:43:55.053826 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0376b9-77a4-4031-82d2-be1e533cb388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:43:55.055562 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0376b9-77a4-4031-82d2-be1e533cb388 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:43:55.055890 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0376b9-77a4-4031-82d2-be1e533cb388 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:43:55.056164 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0376b9-77a4-4031-82d2-be1e533cb388 admin admin] PATH_INFO: `/v3/projects/512450e2a1c54cb684b3748cbeaa7c60/users/76ddf6dd06da4cb0b7c724244356b6db/roles/28763c6e70094116b3a4b2895895fa58` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:43:55.072052 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0376b9-77a4-4031-82d2-be1e533cb388 admin admin] RBAC: Authorizing `identity:create_grant(project_id=512450e2a1c54cb684b3748cbeaa7c60, user_id=76ddf6dd06da4cb0b7c724244356b6db, role_id=28763c6e70094116b3a4b2895895fa58)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:43:55.077185 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0376b9-77a4-4031-82d2-be1e533cb388 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:43:55.149473 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1202/2401] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:43:55 2022] PUT /identity/v3/projects/512450e2a1c54cb684b3748cbeaa7c60/users/76ddf6dd06da4cb0b7c724244356b6db/roles/28763c6e70094116b3a4b2895895fa58 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:43:55.159510 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082e64b5-7a0a-44de-9f04-da95e0ba4d03 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:43:55.159798 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082e64b5-7a0a-44de-9f04-da95e0ba4d03 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:43:55.160075 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082e64b5-7a0a-44de-9f04-da95e0ba4d03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:43:55.187567 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-082e64b5-7a0a-44de-9f04-da95e0ba4d03 None None] MFA Rules not processed for user `76ddf6dd06da4cb0b7c724244356b6db`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:43:55.188785 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-082e64b5-7a0a-44de-9f04-da95e0ba4d03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:43:55.216639 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:43:55.253828 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1200/2402] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:43:55 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:43:55.258282 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0376b9-77a4-4031-82d2-be1e533cb388 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:43:55.319581 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0376b9-77a4-4031-82d2-be1e533cb388 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:43:55.324123 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfd23a1-17af-4af8-a2eb-cbedd2fe56db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:43:55.324778 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfd23a1-17af-4af8-a2eb-cbedd2fe56db admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:43:55.325063 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfd23a1-17af-4af8-a2eb-cbedd2fe56db admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:43:55.325343 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfd23a1-17af-4af8-a2eb-cbedd2fe56db admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:43:55.326077 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfd23a1-17af-4af8-a2eb-cbedd2fe56db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:43:55.327975 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfd23a1-17af-4af8-a2eb-cbedd2fe56db admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:43:55.419523 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:43:55.425927 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1203/2403] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jul 15 19:43:55 2022] POST /identity/v3/projects => generated 381 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:43:55.430371 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082e64b5-7a0a-44de-9f04-da95e0ba4d03 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:43:55.441503 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082e64b5-7a0a-44de-9f04-da95e0ba4d03 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:43:55.498764 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f721a10d-8fe6-4105-95dc-a28fa619c629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:43:55.499722 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f721a10d-8fe6-4105-95dc-a28fa619c629 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:43:55.499971 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f721a10d-8fe6-4105-95dc-a28fa619c629 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:43:55.500250 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f721a10d-8fe6-4105-95dc-a28fa619c629 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:43:55.500950 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f721a10d-8fe6-4105-95dc-a28fa619c629 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:43:55.503082 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f721a10d-8fe6-4105-95dc-a28fa619c629 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:43:55.583798 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1201/2404] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jul 15 19:43:55 2022] POST /identity/v3/users => generated 297 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:43:55.588171 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfd23a1-17af-4af8-a2eb-cbedd2fe56db admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:43:55.599457 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfd23a1-17af-4af8-a2eb-cbedd2fe56db admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:43:55.605222 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1aac0b-373e-4115-8c68-96de15defbe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:43:55.607759 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1aac0b-373e-4115-8c68-96de15defbe4 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:43:55.607759 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1aac0b-373e-4115-8c68-96de15defbe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:43:55.607759 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1aac0b-373e-4115-8c68-96de15defbe4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:43:55.607759 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1aac0b-373e-4115-8c68-96de15defbe4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:43:55.608915 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1aac0b-373e-4115-8c68-96de15defbe4 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:43:55.619784 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1204/2405] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:43:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:43:55.624228 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f721a10d-8fe6-4105-95dc-a28fa619c629 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:43:55.634721 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f721a10d-8fe6-4105-95dc-a28fa619c629 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:43:55.639445 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e580beda-3ecf-4f99-8786-3c7f074a6975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:43:55.640736 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e580beda-3ecf-4f99-8786-3c7f074a6975 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:43:55.640736 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e580beda-3ecf-4f99-8786-3c7f074a6975 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:43:55.640940 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e580beda-3ecf-4f99-8786-3c7f074a6975 admin admin] PATH_INFO: `/v3/projects/ec6af6d5f4e54f57873d0bf8d4a354ba/users/be383f4681d34fba90d085f088291fcc/roles/4318f86ba9f74cd4a0bef1c311276657` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:43:55.657226 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e580beda-3ecf-4f99-8786-3c7f074a6975 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec6af6d5f4e54f57873d0bf8d4a354ba, user_id=be383f4681d34fba90d085f088291fcc, role_id=4318f86ba9f74cd4a0bef1c311276657)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:43:55.660028 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e580beda-3ecf-4f99-8786-3c7f074a6975 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:43:55.773956 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1202/2406] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:43:55 2022] PUT /identity/v3/projects/ec6af6d5f4e54f57873d0bf8d4a354ba/users/be383f4681d34fba90d085f088291fcc/roles/4318f86ba9f74cd4a0bef1c311276657 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:43:55.779132 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1aac0b-373e-4115-8c68-96de15defbe4 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:43:55.789451 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1aac0b-373e-4115-8c68-96de15defbe4 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:43:55.852012 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc97f43c-6472-4c06-b3d6-e6693d647c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:43:55.852012 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc97f43c-6472-4c06-b3d6-e6693d647c22 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:43:55.852012 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc97f43c-6472-4c06-b3d6-e6693d647c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:43:55.852012 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc97f43c-6472-4c06-b3d6-e6693d647c22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:43:55.852603 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc97f43c-6472-4c06-b3d6-e6693d647c22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:43:55.852603 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc97f43c-6472-4c06-b3d6-e6693d647c22 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:43:55.860883 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1205/2407] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:43:55 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:43:55.865030 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e580beda-3ecf-4f99-8786-3c7f074a6975 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:43:55.875645 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e580beda-3ecf-4f99-8786-3c7f074a6975 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:43:55.880331 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de3e451-079b-413b-a861-c4e7ac89fb65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:43:55.881245 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de3e451-079b-413b-a861-c4e7ac89fb65 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:43:55.881526 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de3e451-079b-413b-a861-c4e7ac89fb65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:43:55.881835 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de3e451-079b-413b-a861-c4e7ac89fb65 admin admin] PATH_INFO: `/v3/projects/ec6af6d5f4e54f57873d0bf8d4a354ba/users/be383f4681d34fba90d085f088291fcc/roles/4318f86ba9f74cd4a0bef1c311276657` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:43:55.884312 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de3e451-079b-413b-a861-c4e7ac89fb65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec6af6d5f4e54f57873d0bf8d4a354ba, user_id=be383f4681d34fba90d085f088291fcc, role_id=4318f86ba9f74cd4a0bef1c311276657)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:43:55.886986 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de3e451-079b-413b-a861-c4e7ac89fb65 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:43:55.897263 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1203/2408] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:43:55 2022] PUT /identity/v3/projects/ec6af6d5f4e54f57873d0bf8d4a354ba/users/be383f4681d34fba90d085f088291fcc/roles/4318f86ba9f74cd4a0bef1c311276657 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:43:55.908179 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d6643a-f9d6-4aab-91f3-683b9526be3d None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:43:55.908546 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d6643a-f9d6-4aab-91f3-683b9526be3d None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:43:55.908890 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d6643a-f9d6-4aab-91f3-683b9526be3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:43:55.934932 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-c5d6643a-f9d6-4aab-91f3-683b9526be3d None None] MFA Rules not processed for user `be383f4681d34fba90d085f088291fcc`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:43:55.936166 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-c5d6643a-f9d6-4aab-91f3-683b9526be3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:43:56.004646 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1206/2409] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:43:55 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:43:56.016625 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de3e451-079b-413b-a861-c4e7ac89fb65 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:43:56.018406 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0de3e451-079b-413b-a861-c4e7ac89fb65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:43:56.099756 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de3e451-079b-413b-a861-c4e7ac89fb65 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:43:56.103843 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d9fde4-92db-4444-91ad-ec2c05ab0443 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86c2df3d378490ca7ea497a73c19d40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:43:56.104476 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d9fde4-92db-4444-91ad-ec2c05ab0443 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:43:56.104764 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d9fde4-92db-4444-91ad-ec2c05ab0443 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:43:56.105040 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d9fde4-92db-4444-91ad-ec2c05ab0443 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:43:56.114502 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d9fde4-92db-4444-91ad-ec2c05ab0443 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:43:56.115935 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d9fde4-92db-4444-91ad-ec2c05ab0443 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:43:56.126140 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1204/2410] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:43:56 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:43:56.418623 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d6643a-f9d6-4aab-91f3-683b9526be3d None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:43:56.429915 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d6643a-f9d6-4aab-91f3-683b9526be3d None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:43:56.434506 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fb3934-683d-45c3-a0c7-5c42fe9a0b1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86c2df3d378490ca7ea497a73c19d40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:43:56.435167 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fb3934-683d-45c3-a0c7-5c42fe9a0b1c service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:43:56.435514 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fb3934-683d-45c3-a0c7-5c42fe9a0b1c service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:43:56.435790 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fb3934-683d-45c3-a0c7-5c42fe9a0b1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:43:56.495312 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fb3934-683d-45c3-a0c7-5c42fe9a0b1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:43:56.496754 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fb3934-683d-45c3-a0c7-5c42fe9a0b1c service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:43:56.509609 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1207/2411] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:43:56 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:44:06.121695 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=93,keystone:INSERT=5 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:44:06.502133 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=96 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:44:10.420639 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d9fde4-92db-4444-91ad-ec2c05ab0443 service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:44:10.484276 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d9fde4-92db-4444-91ad-ec2c05ab0443 service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:44:10.488978 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb979880-5f08-4e23-9f5e-f8efc0315a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:44:10.489918 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb979880-5f08-4e23-9f5e-f8efc0315a67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:10.490206 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb979880-5f08-4e23-9f5e-f8efc0315a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:10.490485 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb979880-5f08-4e23-9f5e-f8efc0315a67 admin admin] PATH_INFO: `/v3/users/76ddf6dd06da4cb0b7c724244356b6db` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:10.491751 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb979880-5f08-4e23-9f5e-f8efc0315a67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76ddf6dd06da4cb0b7c724244356b6db)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:44:10.494477 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb979880-5f08-4e23-9f5e-f8efc0315a67 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:44:10.688675 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-fb979880-5f08-4e23-9f5e-f8efc0315a67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76ddf6dd06da4cb0b7c724244356b6db'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:44:10.701600 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-fb979880-5f08-4e23-9f5e-f8efc0315a67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76ddf6dd06da4cb0b7c724244356b6db'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:44:10.788938 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-fb979880-5f08-4e23-9f5e-f8efc0315a67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76ddf6dd06da4cb0b7c724244356b6db'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:44:10.789577 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-fb979880-5f08-4e23-9f5e-f8efc0315a67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76ddf6dd06da4cb0b7c724244356b6db'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:44:10.797169 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-fb979880-5f08-4e23-9f5e-f8efc0315a67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76ddf6dd06da4cb0b7c724244356b6db'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:44:10.803319 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1205/2412] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jul 15 19:44:10 2022] DELETE /identity/v3/users/76ddf6dd06da4cb0b7c724244356b6db => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:44:10.807957 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de48de0-9456-476e-9b1e-3582a71ad04e None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:10.808463 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de48de0-9456-476e-9b1e-3582a71ad04e None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:10.808562 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de48de0-9456-476e-9b1e-3582a71ad04e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:10.826248 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-4de48de0-9456-476e-9b1e-3582a71ad04e None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:44:10.827489 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-4de48de0-9456-476e-9b1e-3582a71ad04e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:44:10.885432 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1208/2413] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:44:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:44:10.895295 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb979880-5f08-4e23-9f5e-f8efc0315a67 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:44:10.896748 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-fb979880-5f08-4e23-9f5e-f8efc0315a67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:44:10.966849 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb979880-5f08-4e23-9f5e-f8efc0315a67 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:44:10.970381 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1d0387-2ce2-4eb2-bef8-5a4fd6813f7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:44:10.971023 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1d0387-2ce2-4eb2-bef8-5a4fd6813f7c service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:10.971301 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1d0387-2ce2-4eb2-bef8-5a4fd6813f7c service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:10.971575 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1d0387-2ce2-4eb2-bef8-5a4fd6813f7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:10.980529 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1d0387-2ce2-4eb2-bef8-5a4fd6813f7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:44:10.982070 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1d0387-2ce2-4eb2-bef8-5a4fd6813f7c service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:44:10.991597 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1206/2414] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:44:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:44:11.493301 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de48de0-9456-476e-9b1e-3582a71ad04e None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:44:11.495079 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-4de48de0-9456-476e-9b1e-3582a71ad04e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:44:11.512455 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de48de0-9456-476e-9b1e-3582a71ad04e None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:44:11.523676 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11e4af8-3138-4709-9851-012477e3fef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:44:11.523676 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e4af8-3138-4709-9851-012477e3fef5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:11.523676 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e4af8-3138-4709-9851-012477e3fef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:11.523676 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e4af8-3138-4709-9851-012477e3fef5 admin admin] PATH_INFO: `/v3/projects/512450e2a1c54cb684b3748cbeaa7c60` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:11.524492 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11e4af8-3138-4709-9851-012477e3fef5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=512450e2a1c54cb684b3748cbeaa7c60)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:44:11.524492 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11e4af8-3138-4709-9851-012477e3fef5 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:44:11.640642 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-d11e4af8-3138-4709-9851-012477e3fef5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '512450e2a1c54cb684b3748cbeaa7c60', 'request_id': 'req-d11e4af8-3138-4709-9851-012477e3fef5'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:44:11.743651 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-d11e4af8-3138-4709-9851-012477e3fef5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '512450e2a1c54cb684b3748cbeaa7c60', 'request_id': 'req-d11e4af8-3138-4709-9851-012477e3fef5'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:44:11.750589 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-d11e4af8-3138-4709-9851-012477e3fef5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '512450e2a1c54cb684b3748cbeaa7c60', 'request_id': 'req-d11e4af8-3138-4709-9851-012477e3fef5'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:44:11.761553 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-d11e4af8-3138-4709-9851-012477e3fef5 admin admin] The token cache is being invalidate because project 512450e2a1c54cb684b3748cbeaa7c60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:44:11.761876 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-d11e4af8-3138-4709-9851-012477e3fef5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:44:11.762522 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-d11e4af8-3138-4709-9851-012477e3fef5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:44:11.764087 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1209/2415] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jul 15 19:44:11 2022] DELETE /identity/v3/projects/512450e2a1c54cb684b3748cbeaa7c60 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:44:11.768128 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1d0387-2ce2-4eb2-bef8-5a4fd6813f7c service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:44:11.778768 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1d0387-2ce2-4eb2-bef8-5a4fd6813f7c service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:44:11.786819 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:44:11.834841 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ee4a1a-49c5-4c27-a1e2-e6b65e9dc3b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:44:11.836266 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ee4a1a-49c5-4c27-a1e2-e6b65e9dc3b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:11.836564 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ee4a1a-49c5-4c27-a1e2-e6b65e9dc3b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:11.836845 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ee4a1a-49c5-4c27-a1e2-e6b65e9dc3b2 admin admin] PATH_INFO: `/v3/users/be383f4681d34fba90d085f088291fcc` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:11.838217 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ee4a1a-49c5-4c27-a1e2-e6b65e9dc3b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be383f4681d34fba90d085f088291fcc)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:44:11.840148 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ee4a1a-49c5-4c27-a1e2-e6b65e9dc3b2 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:44:12.111031 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-36ee4a1a-49c5-4c27-a1e2-e6b65e9dc3b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be383f4681d34fba90d085f088291fcc'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:44:12.122857 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-36ee4a1a-49c5-4c27-a1e2-e6b65e9dc3b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be383f4681d34fba90d085f088291fcc'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:44:12.228249 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-36ee4a1a-49c5-4c27-a1e2-e6b65e9dc3b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be383f4681d34fba90d085f088291fcc'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:44:12.228864 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-36ee4a1a-49c5-4c27-a1e2-e6b65e9dc3b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be383f4681d34fba90d085f088291fcc'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:44:12.236176 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-36ee4a1a-49c5-4c27-a1e2-e6b65e9dc3b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be383f4681d34fba90d085f088291fcc'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:44:12.242220 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1207/2416] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jul 15 19:44:11 2022] DELETE /identity/v3/users/be383f4681d34fba90d085f088291fcc => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:44:12.686946 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11e4af8-3138-4709-9851-012477e3fef5 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:44:12.689111 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-d11e4af8-3138-4709-9851-012477e3fef5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:44:12.703542 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11e4af8-3138-4709-9851-012477e3fef5 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:44:12.764800 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5298aea3-97f7-4633-af85-e571be1e9c24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:44:12.765674 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5298aea3-97f7-4633-af85-e571be1e9c24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:12.765982 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5298aea3-97f7-4633-af85-e571be1e9c24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:12.766258 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5298aea3-97f7-4633-af85-e571be1e9c24 admin admin] PATH_INFO: `/v3/projects/ec6af6d5f4e54f57873d0bf8d4a354ba` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:12.767452 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5298aea3-97f7-4633-af85-e571be1e9c24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec6af6d5f4e54f57873d0bf8d4a354ba)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:44:12.769353 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5298aea3-97f7-4633-af85-e571be1e9c24 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:44:12.893562 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:44:12.910049 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-5298aea3-97f7-4633-af85-e571be1e9c24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec6af6d5f4e54f57873d0bf8d4a354ba', 'request_id': 'req-5298aea3-97f7-4633-af85-e571be1e9c24'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:44:13.040520 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-5298aea3-97f7-4633-af85-e571be1e9c24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec6af6d5f4e54f57873d0bf8d4a354ba', 'request_id': 'req-5298aea3-97f7-4633-af85-e571be1e9c24'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:44:13.047263 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-5298aea3-97f7-4633-af85-e571be1e9c24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec6af6d5f4e54f57873d0bf8d4a354ba', 'request_id': 'req-5298aea3-97f7-4633-af85-e571be1e9c24'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:44:13.057426 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-5298aea3-97f7-4633-af85-e571be1e9c24 admin admin] The token cache is being invalidate because project ec6af6d5f4e54f57873d0bf8d4a354ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:44:13.057718 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-5298aea3-97f7-4633-af85-e571be1e9c24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:44:13.058392 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-5298aea3-97f7-4633-af85-e571be1e9c24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:44:13.059931 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1210/2417] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jul 15 19:44:12 2022] DELETE /identity/v3/projects/ec6af6d5f4e54f57873d0bf8d4a354ba => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:44:13.065056 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0922559-9a40-4155-94f5-a5c8b3238438 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:13.065536 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0922559-9a40-4155-94f5-a5c8b3238438 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:13.065650 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0922559-9a40-4155-94f5-a5c8b3238438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:13.083752 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-d0922559-9a40-4155-94f5-a5c8b3238438 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:44:13.084962 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d0922559-9a40-4155-94f5-a5c8b3238438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:44:13.145226 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1208/2418] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:44:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:44:13.153581 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789d70aa-f5b4-4515-9e27-ccf2560d1a7b None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:13.153902 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789d70aa-f5b4-4515-9e27-ccf2560d1a7b None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:13.154216 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789d70aa-f5b4-4515-9e27-ccf2560d1a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:13.172013 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-789d70aa-f5b4-4515-9e27-ccf2560d1a7b None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:44:13.173214 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-789d70aa-f5b4-4515-9e27-ccf2560d1a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:44:13.181313 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1211/2419] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:44:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:44:13.185404 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0922559-9a40-4155-94f5-a5c8b3238438 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:44:13.195700 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0922559-9a40-4155-94f5-a5c8b3238438 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:44:13.200281 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b945b9d-e5dd-46e4-8aed-bd176955a8f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:44:13.200926 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b945b9d-e5dd-46e4-8aed-bd176955a8f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:13.201203 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b945b9d-e5dd-46e4-8aed-bd176955a8f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:13.201474 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b945b9d-e5dd-46e4-8aed-bd176955a8f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:13.202240 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b945b9d-e5dd-46e4-8aed-bd176955a8f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:44:13.202540 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b945b9d-e5dd-46e4-8aed-bd176955a8f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:44:13.204112 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b945b9d-e5dd-46e4-8aed-bd176955a8f3 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:44:13.215994 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1209/2420] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jul 15 19:44:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:44:13.220121 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789d70aa-f5b4-4515-9e27-ccf2560d1a7b None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:44:13.229088 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789d70aa-f5b4-4515-9e27-ccf2560d1a7b None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:44:13.234133 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8707caf0-a249-4e75-bde7-b783dae9b0ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:44:13.234771 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8707caf0-a249-4e75-bde7-b783dae9b0ac admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:13.235050 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8707caf0-a249-4e75-bde7-b783dae9b0ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:13.235322 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8707caf0-a249-4e75-bde7-b783dae9b0ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:13.236016 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8707caf0-a249-4e75-bde7-b783dae9b0ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:44:13.237921 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8707caf0-a249-4e75-bde7-b783dae9b0ac admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:44:13.344173 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1212/2421] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jul 15 19:44:13 2022] POST /identity/v3/projects => generated 409 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:44:13.348468 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b945b9d-e5dd-46e4-8aed-bd176955a8f3 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:44:13.358380 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b945b9d-e5dd-46e4-8aed-bd176955a8f3 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:44:13.371270 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:44:13.416331 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366c48ae-7d8f-4d53-bfbf-19ddbb1fcadc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:44:13.416927 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366c48ae-7d8f-4d53-bfbf-19ddbb1fcadc admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:13.417210 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366c48ae-7d8f-4d53-bfbf-19ddbb1fcadc admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:13.417484 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366c48ae-7d8f-4d53-bfbf-19ddbb1fcadc admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:13.418213 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366c48ae-7d8f-4d53-bfbf-19ddbb1fcadc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:44:13.420354 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366c48ae-7d8f-4d53-bfbf-19ddbb1fcadc admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:44:13.473938 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1210/2422] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jul 15 19:44:13 2022] POST /identity/v3/users => generated 305 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:44:13.478062 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8707caf0-a249-4e75-bde7-b783dae9b0ac admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:44:13.488554 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8707caf0-a249-4e75-bde7-b783dae9b0ac admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:44:13.494127 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0927381-9478-4073-a041-ac61b956f73d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:44:13.494788 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0927381-9478-4073-a041-ac61b956f73d admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:13.495079 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0927381-9478-4073-a041-ac61b956f73d admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:13.495355 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0927381-9478-4073-a041-ac61b956f73d admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:13.496146 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0927381-9478-4073-a041-ac61b956f73d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:44:13.497757 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0927381-9478-4073-a041-ac61b956f73d admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:44:13.508965 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1213/2423] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:44:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:44:13.518104 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366c48ae-7d8f-4d53-bfbf-19ddbb1fcadc admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:44:13.528644 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366c48ae-7d8f-4d53-bfbf-19ddbb1fcadc admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:44:13.533382 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4542a123-abb2-4797-9caa-4bba1ddb78e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:44:13.534432 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4542a123-abb2-4797-9caa-4bba1ddb78e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:13.534721 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4542a123-abb2-4797-9caa-4bba1ddb78e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:13.534996 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4542a123-abb2-4797-9caa-4bba1ddb78e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:13.535731 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4542a123-abb2-4797-9caa-4bba1ddb78e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:44:13.537336 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4542a123-abb2-4797-9caa-4bba1ddb78e3 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:44:13.547341 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1211/2424] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:44:13 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:44:13.551609 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0927381-9478-4073-a041-ac61b956f73d admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:44:13.562287 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0927381-9478-4073-a041-ac61b956f73d admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:44:13.566940 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75067964-6f9a-4963-bb97-799ea29ff1e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:44:13.567817 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75067964-6f9a-4963-bb97-799ea29ff1e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:13.568100 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75067964-6f9a-4963-bb97-799ea29ff1e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:13.568379 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75067964-6f9a-4963-bb97-799ea29ff1e0 admin admin] PATH_INFO: `/v3/projects/365636f6f27f4131834058ab16689cb2/users/efce01e8fb0e419a9223256f294b968a/roles/28763c6e70094116b3a4b2895895fa58` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:13.592926 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75067964-6f9a-4963-bb97-799ea29ff1e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=365636f6f27f4131834058ab16689cb2, user_id=efce01e8fb0e419a9223256f294b968a, role_id=28763c6e70094116b3a4b2895895fa58)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:44:13.597382 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75067964-6f9a-4963-bb97-799ea29ff1e0 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:44:13.674745 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1214/2425] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:44:13 2022] PUT /identity/v3/projects/365636f6f27f4131834058ab16689cb2/users/efce01e8fb0e419a9223256f294b968a/roles/28763c6e70094116b3a4b2895895fa58 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:44:13.685808 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2044262-dbd8-4cbf-be9a-89b8319d0753 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:13.686102 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2044262-dbd8-4cbf-be9a-89b8319d0753 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:13.686380 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2044262-dbd8-4cbf-be9a-89b8319d0753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:13.714566 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-e2044262-dbd8-4cbf-be9a-89b8319d0753 None None] MFA Rules not processed for user `efce01e8fb0e419a9223256f294b968a`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:44:13.715798 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e2044262-dbd8-4cbf-be9a-89b8319d0753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:44:13.779934 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1212/2426] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:44:13 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:44:13.784554 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75067964-6f9a-4963-bb97-799ea29ff1e0 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:44:13.847344 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75067964-6f9a-4963-bb97-799ea29ff1e0 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:44:13.852591 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f6f166-b2d8-4fa8-9978-27ed2a353302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:44:13.853243 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f6f166-b2d8-4fa8-9978-27ed2a353302 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:13.853525 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f6f166-b2d8-4fa8-9978-27ed2a353302 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:13.853824 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f6f166-b2d8-4fa8-9978-27ed2a353302 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:13.854526 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f6f166-b2d8-4fa8-9978-27ed2a353302 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:44:13.856420 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f6f166-b2d8-4fa8-9978-27ed2a353302 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:44:14.042760 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1215/2427] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jul 15 19:44:13 2022] POST /identity/v3/projects => generated 409 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:44:14.047009 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2044262-dbd8-4cbf-be9a-89b8319d0753 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:44:14.058082 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2044262-dbd8-4cbf-be9a-89b8319d0753 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:44:14.113174 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff0a1be-646c-4bec-a8f0-6b43f5e63f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:44:14.113884 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff0a1be-646c-4bec-a8f0-6b43f5e63f9d admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:14.114124 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff0a1be-646c-4bec-a8f0-6b43f5e63f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:14.114398 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff0a1be-646c-4bec-a8f0-6b43f5e63f9d admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:14.115077 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff0a1be-646c-4bec-a8f0-6b43f5e63f9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:44:14.117492 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff0a1be-646c-4bec-a8f0-6b43f5e63f9d admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:44:14.201994 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1213/2428] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jul 15 19:44:14 2022] POST /identity/v3/users => generated 311 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:44:14.205900 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f6f166-b2d8-4fa8-9978-27ed2a353302 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:44:14.218969 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f6f166-b2d8-4fa8-9978-27ed2a353302 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:44:14.224226 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc70324-ed96-4177-82d1-ab7e6a893b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:44:14.224226 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc70324-ed96-4177-82d1-ab7e6a893b15 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:14.224530 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc70324-ed96-4177-82d1-ab7e6a893b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:14.225045 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc70324-ed96-4177-82d1-ab7e6a893b15 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:14.226640 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc70324-ed96-4177-82d1-ab7e6a893b15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:44:14.228049 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc70324-ed96-4177-82d1-ab7e6a893b15 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:44:14.243681 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1216/2429] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:44:14 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:44:14.247995 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff0a1be-646c-4bec-a8f0-6b43f5e63f9d admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:44:14.259283 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff0a1be-646c-4bec-a8f0-6b43f5e63f9d admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:44:14.264044 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338954a9-e4fa-4500-a378-393903a58bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:44:14.264926 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338954a9-e4fa-4500-a378-393903a58bc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:14.265210 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338954a9-e4fa-4500-a378-393903a58bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:14.265491 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338954a9-e4fa-4500-a378-393903a58bc1 admin admin] PATH_INFO: `/v3/projects/512ede44537549c298ce7fdd23758c46/users/ffee5e351df34dff99c358e0369f3fb1/roles/4318f86ba9f74cd4a0bef1c311276657` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:14.278891 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:44:14.281951 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338954a9-e4fa-4500-a378-393903a58bc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=512ede44537549c298ce7fdd23758c46, user_id=ffee5e351df34dff99c358e0369f3fb1, role_id=4318f86ba9f74cd4a0bef1c311276657)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:44:14.290746 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338954a9-e4fa-4500-a378-393903a58bc1 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:44:14.354955 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1214/2430] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:44:14 2022] PUT /identity/v3/projects/512ede44537549c298ce7fdd23758c46/users/ffee5e351df34dff99c358e0369f3fb1/roles/4318f86ba9f74cd4a0bef1c311276657 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:44:14.359213 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc70324-ed96-4177-82d1-ab7e6a893b15 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:44:14.370011 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc70324-ed96-4177-82d1-ab7e6a893b15 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:44:14.389991 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:44:14.426388 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8a2c75-7b91-4745-b8b7-6fd4a5058df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:44:14.427050 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8a2c75-7b91-4745-b8b7-6fd4a5058df8 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:14.427336 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8a2c75-7b91-4745-b8b7-6fd4a5058df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:14.427690 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8a2c75-7b91-4745-b8b7-6fd4a5058df8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:14.428416 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8a2c75-7b91-4745-b8b7-6fd4a5058df8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:44:14.430596 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8a2c75-7b91-4745-b8b7-6fd4a5058df8 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:44:14.441086 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1217/2431] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:44:14 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:44:14.450797 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338954a9-e4fa-4500-a378-393903a58bc1 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:44:14.461410 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338954a9-e4fa-4500-a378-393903a58bc1 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:44:14.466068 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139397ae-d987-4452-93ad-0b4fbae7296c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:44:14.466952 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139397ae-d987-4452-93ad-0b4fbae7296c admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:14.467235 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139397ae-d987-4452-93ad-0b4fbae7296c admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:14.467512 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139397ae-d987-4452-93ad-0b4fbae7296c admin admin] PATH_INFO: `/v3/projects/512ede44537549c298ce7fdd23758c46/users/ffee5e351df34dff99c358e0369f3fb1/roles/4318f86ba9f74cd4a0bef1c311276657` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:14.469506 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139397ae-d987-4452-93ad-0b4fbae7296c admin admin] RBAC: Authorizing `identity:create_grant(project_id=512ede44537549c298ce7fdd23758c46, user_id=ffee5e351df34dff99c358e0369f3fb1, role_id=4318f86ba9f74cd4a0bef1c311276657)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:44:14.472181 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139397ae-d987-4452-93ad-0b4fbae7296c admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:44:14.482890 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1215/2432] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:44:14 2022] PUT /identity/v3/projects/512ede44537549c298ce7fdd23758c46/users/ffee5e351df34dff99c358e0369f3fb1/roles/4318f86ba9f74cd4a0bef1c311276657 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:44:14.494549 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9fc8d6-5060-4898-85ae-ed25d979a342 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:14.494846 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9fc8d6-5060-4898-85ae-ed25d979a342 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:14.495157 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9fc8d6-5060-4898-85ae-ed25d979a342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:14.523475 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-ca9fc8d6-5060-4898-85ae-ed25d979a342 None None] MFA Rules not processed for user `ffee5e351df34dff99c358e0369f3fb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:44:14.524697 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-ca9fc8d6-5060-4898-85ae-ed25d979a342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:44:14.605138 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1218/2433] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:44:14 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:44:14.616432 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139397ae-d987-4452-93ad-0b4fbae7296c admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:44:14.618770 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-139397ae-d987-4452-93ad-0b4fbae7296c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:44:14.678064 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139397ae-d987-4452-93ad-0b4fbae7296c admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:44:14.682275 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38704095-bf61-4d14-a273-5b92bf9a5785 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86c2df3d378490ca7ea497a73c19d40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:44:14.682921 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38704095-bf61-4d14-a273-5b92bf9a5785 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:14.683200 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38704095-bf61-4d14-a273-5b92bf9a5785 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:14.683477 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38704095-bf61-4d14-a273-5b92bf9a5785 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:14.692292 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38704095-bf61-4d14-a273-5b92bf9a5785 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:44:14.693713 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38704095-bf61-4d14-a273-5b92bf9a5785 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:44:14.703201 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1216/2434] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:44:14 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:44:15.100190 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9fc8d6-5060-4898-85ae-ed25d979a342 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:44:15.109675 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9fc8d6-5060-4898-85ae-ed25d979a342 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:44:15.113214 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345c4bee-f474-4b06-9b17-5edff98a9e12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86c2df3d378490ca7ea497a73c19d40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:44:15.113895 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345c4bee-f474-4b06-9b17-5edff98a9e12 service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:15.114181 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345c4bee-f474-4b06-9b17-5edff98a9e12 service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:15.114456 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345c4bee-f474-4b06-9b17-5edff98a9e12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:15.170018 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345c4bee-f474-4b06-9b17-5edff98a9e12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:44:15.171433 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345c4bee-f474-4b06-9b17-5edff98a9e12 service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:44:15.180238 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1219/2435] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:44:15 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 19:44:24.699162 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=34,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:44:25.176367 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=80 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:44:27.213109 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38704095-bf61-4d14-a273-5b92bf9a5785 service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:44:27.275461 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38704095-bf61-4d14-a273-5b92bf9a5785 service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:44:27.280112 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916a29e7-8a52-486b-a808-6446ef6b68a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:44:27.280972 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916a29e7-8a52-486b-a808-6446ef6b68a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:27.281251 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916a29e7-8a52-486b-a808-6446ef6b68a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:27.281522 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916a29e7-8a52-486b-a808-6446ef6b68a2 admin admin] PATH_INFO: `/v3/users/efce01e8fb0e419a9223256f294b968a` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:27.282724 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916a29e7-8a52-486b-a808-6446ef6b68a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=efce01e8fb0e419a9223256f294b968a)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:44:27.284652 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916a29e7-8a52-486b-a808-6446ef6b68a2 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:44:27.447764 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-916a29e7-8a52-486b-a808-6446ef6b68a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efce01e8fb0e419a9223256f294b968a'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:44:27.459941 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-916a29e7-8a52-486b-a808-6446ef6b68a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efce01e8fb0e419a9223256f294b968a'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:44:27.531481 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-916a29e7-8a52-486b-a808-6446ef6b68a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efce01e8fb0e419a9223256f294b968a'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:44:27.532100 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-916a29e7-8a52-486b-a808-6446ef6b68a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efce01e8fb0e419a9223256f294b968a'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:44:27.539482 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-916a29e7-8a52-486b-a808-6446ef6b68a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efce01e8fb0e419a9223256f294b968a'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:44:27.546243 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1217/2436] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jul 15 19:44:27 2022] DELETE /identity/v3/users/efce01e8fb0e419a9223256f294b968a => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:44:27.550822 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f9cb33-28bc-4cf1-8c19-55bc55aea632 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:27.551310 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f9cb33-28bc-4cf1-8c19-55bc55aea632 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:27.551408 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f9cb33-28bc-4cf1-8c19-55bc55aea632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:27.568180 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-d6f9cb33-28bc-4cf1-8c19-55bc55aea632 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:44:27.569391 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-d6f9cb33-28bc-4cf1-8c19-55bc55aea632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:44:27.627172 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1220/2437] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:44:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:44:27.636802 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916a29e7-8a52-486b-a808-6446ef6b68a2 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:44:27.638393 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-916a29e7-8a52-486b-a808-6446ef6b68a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:44:27.695218 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916a29e7-8a52-486b-a808-6446ef6b68a2 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:44:27.698932 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960a2bd8-79db-47e4-bc23-0f00cc448cec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:44:27.700088 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960a2bd8-79db-47e4-bc23-0f00cc448cec service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:27.700376 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960a2bd8-79db-47e4-bc23-0f00cc448cec service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:27.700652 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960a2bd8-79db-47e4-bc23-0f00cc448cec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:27.710028 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960a2bd8-79db-47e4-bc23-0f00cc448cec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:44:27.711397 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960a2bd8-79db-47e4-bc23-0f00cc448cec service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:44:27.722776 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1218/2438] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:44:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:44:28.213323 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f9cb33-28bc-4cf1-8c19-55bc55aea632 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:44:28.214718 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-d6f9cb33-28bc-4cf1-8c19-55bc55aea632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:44:28.226666 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f9cb33-28bc-4cf1-8c19-55bc55aea632 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:44:28.231391 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65451fb3-b77a-401d-bbb2-a1ee181b98b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:44:28.232269 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65451fb3-b77a-401d-bbb2-a1ee181b98b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:28.232553 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65451fb3-b77a-401d-bbb2-a1ee181b98b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:28.232829 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65451fb3-b77a-401d-bbb2-a1ee181b98b7 admin admin] PATH_INFO: `/v3/projects/365636f6f27f4131834058ab16689cb2` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:28.234052 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65451fb3-b77a-401d-bbb2-a1ee181b98b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=365636f6f27f4131834058ab16689cb2)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:44:28.235962 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65451fb3-b77a-401d-bbb2-a1ee181b98b7 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:44:28.359420 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-65451fb3-b77a-401d-bbb2-a1ee181b98b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '365636f6f27f4131834058ab16689cb2', 'request_id': 'req-65451fb3-b77a-401d-bbb2-a1ee181b98b7'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:44:28.434263 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-65451fb3-b77a-401d-bbb2-a1ee181b98b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '365636f6f27f4131834058ab16689cb2', 'request_id': 'req-65451fb3-b77a-401d-bbb2-a1ee181b98b7'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:44:28.441630 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-65451fb3-b77a-401d-bbb2-a1ee181b98b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '365636f6f27f4131834058ab16689cb2', 'request_id': 'req-65451fb3-b77a-401d-bbb2-a1ee181b98b7'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:44:28.452494 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-65451fb3-b77a-401d-bbb2-a1ee181b98b7 admin admin] The token cache is being invalidate because project 365636f6f27f4131834058ab16689cb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:44:28.452786 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-65451fb3-b77a-401d-bbb2-a1ee181b98b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:44:28.453408 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-65451fb3-b77a-401d-bbb2-a1ee181b98b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:44:28.454988 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1221/2439] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jul 15 19:44:28 2022] DELETE /identity/v3/projects/365636f6f27f4131834058ab16689cb2 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:44:28.458932 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960a2bd8-79db-47e4-bc23-0f00cc448cec service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:44:28.470169 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960a2bd8-79db-47e4-bc23-0f00cc448cec service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:44:28.488115 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:44:28.526121 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607c3361-8a13-4aa3-ac4c-c3bd07fa54d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:44:28.526993 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607c3361-8a13-4aa3-ac4c-c3bd07fa54d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:28.527271 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607c3361-8a13-4aa3-ac4c-c3bd07fa54d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:28.527543 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607c3361-8a13-4aa3-ac4c-c3bd07fa54d1 admin admin] PATH_INFO: `/v3/users/ffee5e351df34dff99c358e0369f3fb1` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:28.528713 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607c3361-8a13-4aa3-ac4c-c3bd07fa54d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffee5e351df34dff99c358e0369f3fb1)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:44:28.530843 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607c3361-8a13-4aa3-ac4c-c3bd07fa54d1 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:44:28.711268 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-607c3361-8a13-4aa3-ac4c-c3bd07fa54d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffee5e351df34dff99c358e0369f3fb1'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:44:28.723561 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-607c3361-8a13-4aa3-ac4c-c3bd07fa54d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffee5e351df34dff99c358e0369f3fb1'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:44:28.805550 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-607c3361-8a13-4aa3-ac4c-c3bd07fa54d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffee5e351df34dff99c358e0369f3fb1'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:44:28.806233 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-607c3361-8a13-4aa3-ac4c-c3bd07fa54d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffee5e351df34dff99c358e0369f3fb1'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:44:28.814530 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-607c3361-8a13-4aa3-ac4c-c3bd07fa54d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffee5e351df34dff99c358e0369f3fb1'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:44:28.820599 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1219/2440] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jul 15 19:44:28 2022] DELETE /identity/v3/users/ffee5e351df34dff99c358e0369f3fb1 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:44:29.516169 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65451fb3-b77a-401d-bbb2-a1ee181b98b7 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:44:29.517670 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-65451fb3-b77a-401d-bbb2-a1ee181b98b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:44:29.532005 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65451fb3-b77a-401d-bbb2-a1ee181b98b7 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:44:29.589169 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f4474a-81f9-41b3-9613-ca3464997636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:44:29.590075 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f4474a-81f9-41b3-9613-ca3464997636 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:29.590360 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f4474a-81f9-41b3-9613-ca3464997636 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:29.590685 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f4474a-81f9-41b3-9613-ca3464997636 admin admin] PATH_INFO: `/v3/projects/512ede44537549c298ce7fdd23758c46` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:29.591877 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f4474a-81f9-41b3-9613-ca3464997636 admin admin] RBAC: Authorizing `identity:delete_project(project_id=512ede44537549c298ce7fdd23758c46)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:44:29.593837 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f4474a-81f9-41b3-9613-ca3464997636 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:44:29.727298 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:44:29.742599 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-d6f4474a-81f9-41b3-9613-ca3464997636 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '512ede44537549c298ce7fdd23758c46', 'request_id': 'req-d6f4474a-81f9-41b3-9613-ca3464997636'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:44:29.851661 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-d6f4474a-81f9-41b3-9613-ca3464997636 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '512ede44537549c298ce7fdd23758c46', 'request_id': 'req-d6f4474a-81f9-41b3-9613-ca3464997636'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:44:29.858406 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-d6f4474a-81f9-41b3-9613-ca3464997636 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '512ede44537549c298ce7fdd23758c46', 'request_id': 'req-d6f4474a-81f9-41b3-9613-ca3464997636'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:44:29.868921 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-d6f4474a-81f9-41b3-9613-ca3464997636 admin admin] The token cache is being invalidate because project 512ede44537549c298ce7fdd23758c46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:44:29.869208 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-d6f4474a-81f9-41b3-9613-ca3464997636 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:44:29.869883 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-d6f4474a-81f9-41b3-9613-ca3464997636 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:44:29.871441 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1222/2441] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jul 15 19:44:29 2022] DELETE /identity/v3/projects/512ede44537549c298ce7fdd23758c46 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:44:29.877148 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e6523f-0628-4419-b454-5de47d777852 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:29.879121 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e6523f-0628-4419-b454-5de47d777852 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:29.879121 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e6523f-0628-4419-b454-5de47d777852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:29.895998 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-78e6523f-0628-4419-b454-5de47d777852 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:44:29.897203 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-78e6523f-0628-4419-b454-5de47d777852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:44:29.956913 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1220/2442] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:44:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:44:29.964817 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0d818d-81b6-4bb2-a185-4fd80dfd209b None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:29.965104 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0d818d-81b6-4bb2-a185-4fd80dfd209b None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:29.965383 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0d818d-81b6-4bb2-a185-4fd80dfd209b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:29.984583 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-cb0d818d-81b6-4bb2-a185-4fd80dfd209b None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:44:29.985830 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-cb0d818d-81b6-4bb2-a185-4fd80dfd209b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:44:29.994368 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1223/2443] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:44:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:44:29.999436 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e6523f-0628-4419-b454-5de47d777852 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:44:30.010027 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e6523f-0628-4419-b454-5de47d777852 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:44:30.015352 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0073beb2-13ec-4fb6-8928-9a1561790c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:44:30.016008 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0073beb2-13ec-4fb6-8928-9a1561790c6f admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:30.016292 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0073beb2-13ec-4fb6-8928-9a1561790c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:30.016565 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0073beb2-13ec-4fb6-8928-9a1561790c6f admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:30.017294 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0073beb2-13ec-4fb6-8928-9a1561790c6f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:44:30.017599 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0073beb2-13ec-4fb6-8928-9a1561790c6f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:44:30.019658 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0073beb2-13ec-4fb6-8928-9a1561790c6f admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:44:30.031613 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1221/2444] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jul 15 19:44:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:44:30.035933 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0d818d-81b6-4bb2-a185-4fd80dfd209b None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:44:30.045757 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0d818d-81b6-4bb2-a185-4fd80dfd209b None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:44:30.050477 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede1f08c-28f6-4938-8758-d8bef858d8d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:44:30.051178 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede1f08c-28f6-4938-8758-d8bef858d8d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:30.051458 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede1f08c-28f6-4938-8758-d8bef858d8d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:30.051731 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede1f08c-28f6-4938-8758-d8bef858d8d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:30.052411 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede1f08c-28f6-4938-8758-d8bef858d8d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:44:30.054339 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede1f08c-28f6-4938-8758-d8bef858d8d2 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:44:30.138107 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1224/2445] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jul 15 19:44:30 2022] POST /identity/v3/projects => generated 379 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:44:30.142406 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0073beb2-13ec-4fb6-8928-9a1561790c6f admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:44:30.152443 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0073beb2-13ec-4fb6-8928-9a1561790c6f admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:44:30.178478 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:44:30.208424 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351c08e8-eb46-4734-b975-e4267e2d60e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:44:30.209096 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351c08e8-eb46-4734-b975-e4267e2d60e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:30.209382 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351c08e8-eb46-4734-b975-e4267e2d60e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:30.209672 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351c08e8-eb46-4734-b975-e4267e2d60e6 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:30.210389 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351c08e8-eb46-4734-b975-e4267e2d60e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:44:30.212312 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351c08e8-eb46-4734-b975-e4267e2d60e6 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:44:30.310105 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1222/2446] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jul 15 19:44:30 2022] POST /identity/v3/users => generated 290 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:44:30.314378 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede1f08c-28f6-4938-8758-d8bef858d8d2 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:44:30.329138 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede1f08c-28f6-4938-8758-d8bef858d8d2 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:44:30.333854 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307755c9-4da0-4ef1-b98c-58282955e6c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:44:30.334520 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307755c9-4da0-4ef1-b98c-58282955e6c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:30.334839 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307755c9-4da0-4ef1-b98c-58282955e6c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:30.335141 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307755c9-4da0-4ef1-b98c-58282955e6c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:30.335871 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307755c9-4da0-4ef1-b98c-58282955e6c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:44:30.337501 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307755c9-4da0-4ef1-b98c-58282955e6c5 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:44:30.348726 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1225/2447] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:44:30 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:44:30.352890 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351c08e8-eb46-4734-b975-e4267e2d60e6 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:44:30.363784 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351c08e8-eb46-4734-b975-e4267e2d60e6 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:44:30.368425 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdfd374-8783-4140-bd55-9d6e2665e9ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:44:30.369075 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdfd374-8783-4140-bd55-9d6e2665e9ca admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:30.369371 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdfd374-8783-4140-bd55-9d6e2665e9ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:30.369627 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdfd374-8783-4140-bd55-9d6e2665e9ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:30.370382 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdfd374-8783-4140-bd55-9d6e2665e9ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:44:30.371973 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdfd374-8783-4140-bd55-9d6e2665e9ca admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:44:30.382333 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1223/2448] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:44:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:44:30.386427 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307755c9-4da0-4ef1-b98c-58282955e6c5 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:44:30.396546 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307755c9-4da0-4ef1-b98c-58282955e6c5 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:44:30.401352 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cb0d4c-37ff-4367-985f-4eb6b3373fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:44:30.402253 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cb0d4c-37ff-4367-985f-4eb6b3373fe3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:30.402529 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cb0d4c-37ff-4367-985f-4eb6b3373fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:30.402799 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cb0d4c-37ff-4367-985f-4eb6b3373fe3 admin admin] PATH_INFO: `/v3/projects/6d94325cfd6b41ec86e08ee0a7c675e3/users/df4a4b1063574fc4a44d7f16ae8aa102/roles/28763c6e70094116b3a4b2895895fa58` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:30.418005 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cb0d4c-37ff-4367-985f-4eb6b3373fe3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d94325cfd6b41ec86e08ee0a7c675e3, user_id=df4a4b1063574fc4a44d7f16ae8aa102, role_id=28763c6e70094116b3a4b2895895fa58)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:44:30.421287 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cb0d4c-37ff-4367-985f-4eb6b3373fe3 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:44:30.493380 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1226/2449] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:44:30 2022] PUT /identity/v3/projects/6d94325cfd6b41ec86e08ee0a7c675e3/users/df4a4b1063574fc4a44d7f16ae8aa102/roles/28763c6e70094116b3a4b2895895fa58 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:44:30.503995 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd516114-c03d-483b-a4ef-30f71e1785d0 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:30.504292 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd516114-c03d-483b-a4ef-30f71e1785d0 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:30.504573 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd516114-c03d-483b-a4ef-30f71e1785d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:30.531425 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-dd516114-c03d-483b-a4ef-30f71e1785d0 None None] MFA Rules not processed for user `df4a4b1063574fc4a44d7f16ae8aa102`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:44:30.532639 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-dd516114-c03d-483b-a4ef-30f71e1785d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:44:30.596270 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1224/2450] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:44:30 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:44:30.600656 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cb0d4c-37ff-4367-985f-4eb6b3373fe3 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:44:30.664618 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cb0d4c-37ff-4367-985f-4eb6b3373fe3 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:44:30.669282 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9058434-f03a-4c53-93fb-335614925b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:44:30.669983 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9058434-f03a-4c53-93fb-335614925b04 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:30.670792 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9058434-f03a-4c53-93fb-335614925b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:30.671072 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9058434-f03a-4c53-93fb-335614925b04 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:30.671774 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9058434-f03a-4c53-93fb-335614925b04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:44:30.673682 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9058434-f03a-4c53-93fb-335614925b04 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:44:30.807336 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1227/2451] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jul 15 19:44:30 2022] POST /identity/v3/projects => generated 379 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:44:30.811443 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd516114-c03d-483b-a4ef-30f71e1785d0 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:44:30.822230 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd516114-c03d-483b-a4ef-30f71e1785d0 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:44:30.878296 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb94224-dab6-493d-8f35-49a12b404139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:44:30.878953 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb94224-dab6-493d-8f35-49a12b404139 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:30.879231 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb94224-dab6-493d-8f35-49a12b404139 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:30.879509 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb94224-dab6-493d-8f35-49a12b404139 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:30.880195 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb94224-dab6-493d-8f35-49a12b404139 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:44:30.882163 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb94224-dab6-493d-8f35-49a12b404139 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:44:31.006286 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1225/2452] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jul 15 19:44:30 2022] POST /identity/v3/users => generated 296 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:44:31.010251 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9058434-f03a-4c53-93fb-335614925b04 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:44:31.020988 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9058434-f03a-4c53-93fb-335614925b04 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:44:31.025515 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5316a280-45b8-4a0b-bc03-ae7953869339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:44:31.026207 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5316a280-45b8-4a0b-bc03-ae7953869339 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:31.026492 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5316a280-45b8-4a0b-bc03-ae7953869339 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:31.026772 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5316a280-45b8-4a0b-bc03-ae7953869339 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:31.027501 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5316a280-45b8-4a0b-bc03-ae7953869339 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:44:31.029097 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5316a280-45b8-4a0b-bc03-ae7953869339 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:44:31.040424 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1228/2453] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:44:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:44:31.044731 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb94224-dab6-493d-8f35-49a12b404139 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:44:31.054831 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb94224-dab6-493d-8f35-49a12b404139 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:44:31.059406 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa6aa86-2290-4f20-a86b-a532c2202f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:44:31.060281 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa6aa86-2290-4f20-a86b-a532c2202f08 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:31.060564 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa6aa86-2290-4f20-a86b-a532c2202f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:31.061341 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa6aa86-2290-4f20-a86b-a532c2202f08 admin admin] PATH_INFO: `/v3/projects/a85406dccb7e4eac98dd0956bf25c17b/users/4f99a0e234354f34bbef7b7b82838d41/roles/4318f86ba9f74cd4a0bef1c311276657` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:31.078485 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa6aa86-2290-4f20-a86b-a532c2202f08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a85406dccb7e4eac98dd0956bf25c17b, user_id=4f99a0e234354f34bbef7b7b82838d41, role_id=4318f86ba9f74cd4a0bef1c311276657)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:44:31.081200 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa6aa86-2290-4f20-a86b-a532c2202f08 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:44:31.176383 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1226/2454] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:44:31 2022] PUT /identity/v3/projects/a85406dccb7e4eac98dd0956bf25c17b/users/4f99a0e234354f34bbef7b7b82838d41/roles/4318f86ba9f74cd4a0bef1c311276657 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:44:31.180519 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5316a280-45b8-4a0b-bc03-ae7953869339 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:44:31.191138 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5316a280-45b8-4a0b-bc03-ae7953869339 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:44:31.211379 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:44:31.247425 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19179733-59b9-487e-a03a-855d25e1116a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:44:31.248789 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19179733-59b9-487e-a03a-855d25e1116a admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:31.249071 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19179733-59b9-487e-a03a-855d25e1116a admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:31.249458 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19179733-59b9-487e-a03a-855d25e1116a admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:31.250184 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19179733-59b9-487e-a03a-855d25e1116a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:44:31.252262 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19179733-59b9-487e-a03a-855d25e1116a admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:44:31.262249 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1229/2455] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:44:31 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:44:31.267136 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa6aa86-2290-4f20-a86b-a532c2202f08 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:44:31.272611 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:44:31.277876 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa6aa86-2290-4f20-a86b-a532c2202f08 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:44:31.282438 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27b5d37-efd1-441f-a672-eada714d96ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:44:31.283315 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b5d37-efd1-441f-a672-eada714d96ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:31.283592 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b5d37-efd1-441f-a672-eada714d96ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:31.283923 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b5d37-efd1-441f-a672-eada714d96ba admin admin] PATH_INFO: `/v3/projects/a85406dccb7e4eac98dd0956bf25c17b/users/4f99a0e234354f34bbef7b7b82838d41/roles/4318f86ba9f74cd4a0bef1c311276657` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:31.286858 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27b5d37-efd1-441f-a672-eada714d96ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=a85406dccb7e4eac98dd0956bf25c17b, user_id=4f99a0e234354f34bbef7b7b82838d41, role_id=4318f86ba9f74cd4a0bef1c311276657)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:44:31.289475 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27b5d37-efd1-441f-a672-eada714d96ba admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:44:31.299536 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1227/2456] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:44:31 2022] PUT /identity/v3/projects/a85406dccb7e4eac98dd0956bf25c17b/users/4f99a0e234354f34bbef7b7b82838d41/roles/4318f86ba9f74cd4a0bef1c311276657 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:44:31.311260 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6e053e-33a2-47cf-be06-a9130a322ce1 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:31.311567 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6e053e-33a2-47cf-be06-a9130a322ce1 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:31.311845 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6e053e-33a2-47cf-be06-a9130a322ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:31.337995 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-9f6e053e-33a2-47cf-be06-a9130a322ce1 None None] MFA Rules not processed for user `4f99a0e234354f34bbef7b7b82838d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:44:31.339247 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-9f6e053e-33a2-47cf-be06-a9130a322ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:44:31.419354 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1230/2457] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:44:31 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:44:31.430548 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27b5d37-efd1-441f-a672-eada714d96ba admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:44:31.437888 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-f27b5d37-efd1-441f-a672-eada714d96ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:44:31.489701 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27b5d37-efd1-441f-a672-eada714d96ba admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:44:31.493370 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa33530a-fc40-47f5-bf31-3b5850cf7568 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86c2df3d378490ca7ea497a73c19d40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:44:31.494041 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa33530a-fc40-47f5-bf31-3b5850cf7568 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:31.494321 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa33530a-fc40-47f5-bf31-3b5850cf7568 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:31.494595 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa33530a-fc40-47f5-bf31-3b5850cf7568 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:31.504706 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa33530a-fc40-47f5-bf31-3b5850cf7568 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:44:31.506092 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa33530a-fc40-47f5-bf31-3b5850cf7568 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:44:31.515912 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1228/2458] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:44:31 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:44:32.060186 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6e053e-33a2-47cf-be06-a9130a322ce1 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:44:32.122742 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6e053e-33a2-47cf-be06-a9130a322ce1 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:44:32.128133 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ab7338-4361-44fb-82c4-f1c5afb86e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:44:32.129002 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ab7338-4361-44fb-82c4-f1c5afb86e54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:32.129283 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ab7338-4361-44fb-82c4-f1c5afb86e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:32.129556 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ab7338-4361-44fb-82c4-f1c5afb86e54 admin admin] PATH_INFO: `/v3/users/df4a4b1063574fc4a44d7f16ae8aa102` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:32.130769 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ab7338-4361-44fb-82c4-f1c5afb86e54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df4a4b1063574fc4a44d7f16ae8aa102)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:44:32.132659 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ab7338-4361-44fb-82c4-f1c5afb86e54 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:44:32.286856 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-92ab7338-4361-44fb-82c4-f1c5afb86e54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df4a4b1063574fc4a44d7f16ae8aa102'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:44:32.288495 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:44:32.300588 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-92ab7338-4361-44fb-82c4-f1c5afb86e54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df4a4b1063574fc4a44d7f16ae8aa102'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:44:32.353591 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-92ab7338-4361-44fb-82c4-f1c5afb86e54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df4a4b1063574fc4a44d7f16ae8aa102'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:44:32.354245 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-92ab7338-4361-44fb-82c4-f1c5afb86e54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df4a4b1063574fc4a44d7f16ae8aa102'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:44:32.354876 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-92ab7338-4361-44fb-82c4-f1c5afb86e54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df4a4b1063574fc4a44d7f16ae8aa102'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:44:32.367784 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1231/2459] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jul 15 19:44:32 2022] DELETE /identity/v3/users/df4a4b1063574fc4a44d7f16ae8aa102 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:44:32.372421 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71617d56-035a-4477-ba0e-a379cf8c43b0 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:32.373156 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71617d56-035a-4477-ba0e-a379cf8c43b0 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:32.373815 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71617d56-035a-4477-ba0e-a379cf8c43b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:32.391712 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-71617d56-035a-4477-ba0e-a379cf8c43b0 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:44:32.393122 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-71617d56-035a-4477-ba0e-a379cf8c43b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:44:32.452102 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1229/2460] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:44:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:44:32.466804 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ab7338-4361-44fb-82c4-f1c5afb86e54 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:44:32.468250 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-92ab7338-4361-44fb-82c4-f1c5afb86e54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:44:32.526829 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ab7338-4361-44fb-82c4-f1c5afb86e54 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:44:32.531238 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213530a3-7ee4-46c1-945d-12e264080e88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:44:32.531897 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213530a3-7ee4-46c1-945d-12e264080e88 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:32.532219 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213530a3-7ee4-46c1-945d-12e264080e88 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:32.532499 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213530a3-7ee4-46c1-945d-12e264080e88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:32.542189 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213530a3-7ee4-46c1-945d-12e264080e88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:44:32.543779 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213530a3-7ee4-46c1-945d-12e264080e88 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:44:32.554516 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1232/2461] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:44:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:44:33.007057 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71617d56-035a-4477-ba0e-a379cf8c43b0 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:44:33.009473 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-71617d56-035a-4477-ba0e-a379cf8c43b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:44:33.022132 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71617d56-035a-4477-ba0e-a379cf8c43b0 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:44:33.027269 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda807fb-94d2-4b39-a2e6-cb577266f237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:44:33.028146 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda807fb-94d2-4b39-a2e6-cb577266f237 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:33.028427 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda807fb-94d2-4b39-a2e6-cb577266f237 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:33.028698 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda807fb-94d2-4b39-a2e6-cb577266f237 admin admin] PATH_INFO: `/v3/projects/6d94325cfd6b41ec86e08ee0a7c675e3` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:33.030502 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda807fb-94d2-4b39-a2e6-cb577266f237 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d94325cfd6b41ec86e08ee0a7c675e3)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:44:33.032544 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda807fb-94d2-4b39-a2e6-cb577266f237 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:44:33.166995 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-fda807fb-94d2-4b39-a2e6-cb577266f237 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d94325cfd6b41ec86e08ee0a7c675e3', 'request_id': 'req-fda807fb-94d2-4b39-a2e6-cb577266f237'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:44:33.239513 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-fda807fb-94d2-4b39-a2e6-cb577266f237 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d94325cfd6b41ec86e08ee0a7c675e3', 'request_id': 'req-fda807fb-94d2-4b39-a2e6-cb577266f237'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:44:33.245799 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-fda807fb-94d2-4b39-a2e6-cb577266f237 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d94325cfd6b41ec86e08ee0a7c675e3', 'request_id': 'req-fda807fb-94d2-4b39-a2e6-cb577266f237'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:44:33.258383 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-fda807fb-94d2-4b39-a2e6-cb577266f237 admin admin] The token cache is being invalidate because project 6d94325cfd6b41ec86e08ee0a7c675e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:44:33.258694 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-fda807fb-94d2-4b39-a2e6-cb577266f237 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:44:33.259345 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-fda807fb-94d2-4b39-a2e6-cb577266f237 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:44:33.260990 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1230/2462] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jul 15 19:44:33 2022] DELETE /identity/v3/projects/6d94325cfd6b41ec86e08ee0a7c675e3 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:44:33.265034 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213530a3-7ee4-46c1-945d-12e264080e88 service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:44:33.274666 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213530a3-7ee4-46c1-945d-12e264080e88 service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:44:33.330742 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a71b07-65dd-4bca-a989-7960614f88d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:44:33.331607 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a71b07-65dd-4bca-a989-7960614f88d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:33.331892 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a71b07-65dd-4bca-a989-7960614f88d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:33.332171 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a71b07-65dd-4bca-a989-7960614f88d7 admin admin] PATH_INFO: `/v3/users/4f99a0e234354f34bbef7b7b82838d41` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:33.333352 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a71b07-65dd-4bca-a989-7960614f88d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f99a0e234354f34bbef7b7b82838d41)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:44:33.335299 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a71b07-65dd-4bca-a989-7960614f88d7 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:44:33.545691 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-c1a71b07-65dd-4bca-a989-7960614f88d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f99a0e234354f34bbef7b7b82838d41'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:44:33.557695 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-c1a71b07-65dd-4bca-a989-7960614f88d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f99a0e234354f34bbef7b7b82838d41'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:44:33.561919 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:44:33.645741 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-c1a71b07-65dd-4bca-a989-7960614f88d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f99a0e234354f34bbef7b7b82838d41'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:44:33.649720 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-c1a71b07-65dd-4bca-a989-7960614f88d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f99a0e234354f34bbef7b7b82838d41'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:44:33.649720 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-c1a71b07-65dd-4bca-a989-7960614f88d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f99a0e234354f34bbef7b7b82838d41'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:44:33.659793 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1233/2463] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jul 15 19:44:33 2022] DELETE /identity/v3/users/4f99a0e234354f34bbef7b7b82838d41 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:44:34.081911 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda807fb-94d2-4b39-a2e6-cb577266f237 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:44:34.083444 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-fda807fb-94d2-4b39-a2e6-cb577266f237 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:44:34.097205 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda807fb-94d2-4b39-a2e6-cb577266f237 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:44:34.110013 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:44:34.156096 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6725e643-f642-445f-9f96-2344c65d2c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:44:34.156957 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6725e643-f642-445f-9f96-2344c65d2c3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:34.157237 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6725e643-f642-445f-9f96-2344c65d2c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:34.157511 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6725e643-f642-445f-9f96-2344c65d2c3e admin admin] PATH_INFO: `/v3/projects/a85406dccb7e4eac98dd0956bf25c17b` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:34.158724 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6725e643-f642-445f-9f96-2344c65d2c3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a85406dccb7e4eac98dd0956bf25c17b)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:44:34.160622 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6725e643-f642-445f-9f96-2344c65d2c3e admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:44:34.273982 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6725e643-f642-445f-9f96-2344c65d2c3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a85406dccb7e4eac98dd0956bf25c17b', 'request_id': 'req-6725e643-f642-445f-9f96-2344c65d2c3e'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:44:34.331075 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6725e643-f642-445f-9f96-2344c65d2c3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a85406dccb7e4eac98dd0956bf25c17b', 'request_id': 'req-6725e643-f642-445f-9f96-2344c65d2c3e'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:44:34.336778 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6725e643-f642-445f-9f96-2344c65d2c3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a85406dccb7e4eac98dd0956bf25c17b', 'request_id': 'req-6725e643-f642-445f-9f96-2344c65d2c3e'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:44:34.347706 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6725e643-f642-445f-9f96-2344c65d2c3e admin admin] The token cache is being invalidate because project a85406dccb7e4eac98dd0956bf25c17b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:44:34.347994 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6725e643-f642-445f-9f96-2344c65d2c3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:44:34.348616 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6725e643-f642-445f-9f96-2344c65d2c3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:44:34.350187 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1231/2464] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jul 15 19:44:34 2022] DELETE /identity/v3/projects/a85406dccb7e4eac98dd0956bf25c17b => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:44:34.355618 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69469ab2-4d93-4e73-ad69-8f08d508c321 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:34.356108 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69469ab2-4d93-4e73-ad69-8f08d508c321 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:34.356215 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69469ab2-4d93-4e73-ad69-8f08d508c321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:34.374943 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-69469ab2-4d93-4e73-ad69-8f08d508c321 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:44:34.376162 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-69469ab2-4d93-4e73-ad69-8f08d508c321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:44:34.436315 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1234/2465] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:44:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:44:34.443629 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8482da-8b20-47b0-ad1d-110ac85a4878 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:34.443919 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8482da-8b20-47b0-ad1d-110ac85a4878 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:34.444201 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8482da-8b20-47b0-ad1d-110ac85a4878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:34.462423 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-cc8482da-8b20-47b0-ad1d-110ac85a4878 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:44:34.463624 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-cc8482da-8b20-47b0-ad1d-110ac85a4878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:44:34.471634 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1232/2466] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:44:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:44:34.475810 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69469ab2-4d93-4e73-ad69-8f08d508c321 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:44:34.485983 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69469ab2-4d93-4e73-ad69-8f08d508c321 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:44:34.490716 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b63be27-d529-4caa-b45d-2d378cebb642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:44:34.491412 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b63be27-d529-4caa-b45d-2d378cebb642 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:34.491764 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b63be27-d529-4caa-b45d-2d378cebb642 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:34.491965 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b63be27-d529-4caa-b45d-2d378cebb642 admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:34.492686 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b63be27-d529-4caa-b45d-2d378cebb642 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:44:34.493114 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b63be27-d529-4caa-b45d-2d378cebb642 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:44:34.494764 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b63be27-d529-4caa-b45d-2d378cebb642 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:44:34.507530 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1235/2467] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jul 15 19:44:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:44:34.511578 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8482da-8b20-47b0-ad1d-110ac85a4878 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:44:34.520933 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8482da-8b20-47b0-ad1d-110ac85a4878 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:44:34.526072 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1214e8-0123-4714-a53f-bfd521e1abbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:44:34.526749 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1214e8-0123-4714-a53f-bfd521e1abbd admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:34.527060 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1214e8-0123-4714-a53f-bfd521e1abbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:34.527341 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1214e8-0123-4714-a53f-bfd521e1abbd admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:34.528029 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1214e8-0123-4714-a53f-bfd521e1abbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:44:34.530040 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1214e8-0123-4714-a53f-bfd521e1abbd admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:44:34.641922 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1233/2468] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jul 15 19:44:34 2022] POST /identity/v3/projects => generated 383 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:44:34.646172 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b63be27-d529-4caa-b45d-2d378cebb642 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:44:34.656408 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b63be27-d529-4caa-b45d-2d378cebb642 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:44:34.712630 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847ac8de-e02e-4c2f-a16b-961256e56ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:44:34.713291 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ac8de-e02e-4c2f-a16b-961256e56ff4 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:34.713570 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ac8de-e02e-4c2f-a16b-961256e56ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:34.713871 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ac8de-e02e-4c2f-a16b-961256e56ff4 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:34.714586 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847ac8de-e02e-4c2f-a16b-961256e56ff4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:44:34.716497 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847ac8de-e02e-4c2f-a16b-961256e56ff4 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:44:34.840445 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1236/2469] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jul 15 19:44:34 2022] POST /identity/v3/users => generated 292 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:44:34.845116 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1214e8-0123-4714-a53f-bfd521e1abbd admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:44:34.855611 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1214e8-0123-4714-a53f-bfd521e1abbd admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:44:34.860868 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e911b2c3-71d3-47f4-9284-024cd26439a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:44:34.861525 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e911b2c3-71d3-47f4-9284-024cd26439a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:34.861837 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e911b2c3-71d3-47f4-9284-024cd26439a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:34.862118 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e911b2c3-71d3-47f4-9284-024cd26439a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:34.862851 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e911b2c3-71d3-47f4-9284-024cd26439a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:44:34.864437 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e911b2c3-71d3-47f4-9284-024cd26439a0 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:44:34.876047 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1234/2470] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:44:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:44:34.880002 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847ac8de-e02e-4c2f-a16b-961256e56ff4 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:44:34.890779 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847ac8de-e02e-4c2f-a16b-961256e56ff4 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:44:34.898101 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf026af-e621-4745-b313-0de38ed12b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:44:34.898101 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf026af-e621-4745-b313-0de38ed12b93 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:34.898101 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf026af-e621-4745-b313-0de38ed12b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:34.898101 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf026af-e621-4745-b313-0de38ed12b93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:34.898540 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf026af-e621-4745-b313-0de38ed12b93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:44:34.899075 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf026af-e621-4745-b313-0de38ed12b93 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:44:34.909664 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1237/2471] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:44:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:44:34.913720 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e911b2c3-71d3-47f4-9284-024cd26439a0 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:44:34.923735 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e911b2c3-71d3-47f4-9284-024cd26439a0 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:44:34.928396 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a925600-98f8-472e-96e6-ffcfaed7c238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:44:34.929276 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a925600-98f8-472e-96e6-ffcfaed7c238 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:34.929558 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a925600-98f8-472e-96e6-ffcfaed7c238 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:34.929867 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a925600-98f8-472e-96e6-ffcfaed7c238 admin admin] PATH_INFO: `/v3/projects/68b154c5961f41d09a61096db1fd5000/users/a81dac16310b418fa392568f64b84e40/roles/28763c6e70094116b3a4b2895895fa58` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:34.944720 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a925600-98f8-472e-96e6-ffcfaed7c238 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68b154c5961f41d09a61096db1fd5000, user_id=a81dac16310b418fa392568f64b84e40, role_id=28763c6e70094116b3a4b2895895fa58)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:44:34.947430 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a925600-98f8-472e-96e6-ffcfaed7c238 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:44:35.011286 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1235/2472] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:44:34 2022] PUT /identity/v3/projects/68b154c5961f41d09a61096db1fd5000/users/a81dac16310b418fa392568f64b84e40/roles/28763c6e70094116b3a4b2895895fa58 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:44:35.023374 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766e4d97-383d-4e1d-b2a3-262eef6bd9a3 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:35.024490 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766e4d97-383d-4e1d-b2a3-262eef6bd9a3 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:35.024490 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766e4d97-383d-4e1d-b2a3-262eef6bd9a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:35.051395 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-766e4d97-383d-4e1d-b2a3-262eef6bd9a3 None None] MFA Rules not processed for user `a81dac16310b418fa392568f64b84e40`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:44:35.052619 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-766e4d97-383d-4e1d-b2a3-262eef6bd9a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:44:35.093438 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:44:35.116645 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1238/2473] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:44:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:44:35.129163 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a925600-98f8-472e-96e6-ffcfaed7c238 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:44:35.130960 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-6a925600-98f8-472e-96e6-ffcfaed7c238 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:44:35.176393 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:44:35.188052 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a925600-98f8-472e-96e6-ffcfaed7c238 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:44:35.191733 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be77f65-13eb-4026-9a8e-1095ba51431d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86c2df3d378490ca7ea497a73c19d40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:44:35.192377 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be77f65-13eb-4026-9a8e-1095ba51431d service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:35.192665 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be77f65-13eb-4026-9a8e-1095ba51431d service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:35.192945 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be77f65-13eb-4026-9a8e-1095ba51431d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:35.202266 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be77f65-13eb-4026-9a8e-1095ba51431d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:44:35.203673 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be77f65-13eb-4026-9a8e-1095ba51431d service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:44:35.212673 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1236/2474] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:44:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:44:35.231686 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766e4d97-383d-4e1d-b2a3-262eef6bd9a3 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:44:35.293065 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766e4d97-383d-4e1d-b2a3-262eef6bd9a3 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:44:35.298189 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa02fa1f-a728-43a3-89cd-f65554a988f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:44:35.299064 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa02fa1f-a728-43a3-89cd-f65554a988f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:35.299360 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa02fa1f-a728-43a3-89cd-f65554a988f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:35.299638 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa02fa1f-a728-43a3-89cd-f65554a988f6 admin admin] PATH_INFO: `/v3/users/a81dac16310b418fa392568f64b84e40` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:35.300819 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa02fa1f-a728-43a3-89cd-f65554a988f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a81dac16310b418fa392568f64b84e40)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:44:35.302769 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa02fa1f-a728-43a3-89cd-f65554a988f6 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:44:35.516771 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-fa02fa1f-a728-43a3-89cd-f65554a988f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a81dac16310b418fa392568f64b84e40'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:44:35.528505 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-fa02fa1f-a728-43a3-89cd-f65554a988f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a81dac16310b418fa392568f64b84e40'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:44:35.593939 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-fa02fa1f-a728-43a3-89cd-f65554a988f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a81dac16310b418fa392568f64b84e40'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:44:35.594568 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-fa02fa1f-a728-43a3-89cd-f65554a988f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a81dac16310b418fa392568f64b84e40'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:44:35.595209 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-fa02fa1f-a728-43a3-89cd-f65554a988f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a81dac16310b418fa392568f64b84e40'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:44:35.608589 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1239/2475] 10.222.0.33 () {58 vars in 1223 bytes} [Fri Jul 15 19:44:35 2022] DELETE /identity/v3/users/a81dac16310b418fa392568f64b84e40 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:44:35.613502 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291de7db-9a39-47df-b28e-6450c293be33 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:35.613999 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291de7db-9a39-47df-b28e-6450c293be33 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:35.614225 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291de7db-9a39-47df-b28e-6450c293be33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:35.631761 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-291de7db-9a39-47df-b28e-6450c293be33 None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:44:35.633085 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-291de7db-9a39-47df-b28e-6450c293be33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:44:35.692682 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1237/2476] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:44:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:44:35.702457 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa02fa1f-a728-43a3-89cd-f65554a988f6 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:44:35.703926 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-fa02fa1f-a728-43a3-89cd-f65554a988f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:44:35.762646 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa02fa1f-a728-43a3-89cd-f65554a988f6 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:44:35.766312 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8765598-f689-4d27-9fe8-3c2a16c87520 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:44:35.766975 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8765598-f689-4d27-9fe8-3c2a16c87520 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:35.767254 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8765598-f689-4d27-9fe8-3c2a16c87520 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:35.767530 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8765598-f689-4d27-9fe8-3c2a16c87520 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:35.776879 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8765598-f689-4d27-9fe8-3c2a16c87520 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:44:35.778347 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8765598-f689-4d27-9fe8-3c2a16c87520 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:44:35.788520 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1240/2477] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:44:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:44:36.302508 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291de7db-9a39-47df-b28e-6450c293be33 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:44:36.304070 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-291de7db-9a39-47df-b28e-6450c293be33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:44:36.316615 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291de7db-9a39-47df-b28e-6450c293be33 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:44:36.321985 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af1847c-51fd-434b-a66a-3b3d4ca0b2a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:44:36.322870 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af1847c-51fd-434b-a66a-3b3d4ca0b2a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:36.323291 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af1847c-51fd-434b-a66a-3b3d4ca0b2a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:36.323579 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af1847c-51fd-434b-a66a-3b3d4ca0b2a8 admin admin] PATH_INFO: `/v3/projects/68b154c5961f41d09a61096db1fd5000` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:36.324800 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af1847c-51fd-434b-a66a-3b3d4ca0b2a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68b154c5961f41d09a61096db1fd5000)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:44:36.329120 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af1847c-51fd-434b-a66a-3b3d4ca0b2a8 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:44:36.453081 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-1af1847c-51fd-434b-a66a-3b3d4ca0b2a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68b154c5961f41d09a61096db1fd5000', 'request_id': 'req-1af1847c-51fd-434b-a66a-3b3d4ca0b2a8'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:44:36.559790 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-1af1847c-51fd-434b-a66a-3b3d4ca0b2a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68b154c5961f41d09a61096db1fd5000', 'request_id': 'req-1af1847c-51fd-434b-a66a-3b3d4ca0b2a8'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:44:36.565917 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-1af1847c-51fd-434b-a66a-3b3d4ca0b2a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68b154c5961f41d09a61096db1fd5000', 'request_id': 'req-1af1847c-51fd-434b-a66a-3b3d4ca0b2a8'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:44:36.577014 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-1af1847c-51fd-434b-a66a-3b3d4ca0b2a8 admin admin] The token cache is being invalidate because project 68b154c5961f41d09a61096db1fd5000 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:44:36.577296 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-1af1847c-51fd-434b-a66a-3b3d4ca0b2a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:44:36.577967 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-1af1847c-51fd-434b-a66a-3b3d4ca0b2a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:44:36.579519 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1238/2478] 10.222.0.33 () {58 vars in 1232 bytes} [Fri Jul 15 19:44:36 2022] DELETE /identity/v3/projects/68b154c5961f41d09a61096db1fd5000 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:44:36.584405 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac0f935-3593-43c1-a752-059d924dbe7a None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:36.584874 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac0f935-3593-43c1-a752-059d924dbe7a None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:36.584979 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac0f935-3593-43c1-a752-059d924dbe7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:36.593380 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:44:36.605021 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-2ac0f935-3593-43c1-a752-059d924dbe7a None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:44:36.609200 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-2ac0f935-3593-43c1-a752-059d924dbe7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:44:36.668794 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1241/2479] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:44:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:44:36.677749 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1518d5ab-cd53-402a-83f8-7e15bfade17b None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:36.678726 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1518d5ab-cd53-402a-83f8-7e15bfade17b None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:36.678726 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1518d5ab-cd53-402a-83f8-7e15bfade17b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:36.698319 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-1518d5ab-cd53-402a-83f8-7e15bfade17b None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:44:36.707777 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-1518d5ab-cd53-402a-83f8-7e15bfade17b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:44:36.716413 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1239/2480] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:44:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:44:36.721172 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac0f935-3593-43c1-a752-059d924dbe7a None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:44:36.731278 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac0f935-3593-43c1-a752-059d924dbe7a None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:44:36.737705 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd7eb95-5371-4dcd-8b1f-3ca8e46b9392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:44:36.738396 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd7eb95-5371-4dcd-8b1f-3ca8e46b9392 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:36.747706 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd7eb95-5371-4dcd-8b1f-3ca8e46b9392 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:36.747706 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd7eb95-5371-4dcd-8b1f-3ca8e46b9392 admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:36.747706 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd7eb95-5371-4dcd-8b1f-3ca8e46b9392 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:44:36.747706 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd7eb95-5371-4dcd-8b1f-3ca8e46b9392 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:44:36.747706 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd7eb95-5371-4dcd-8b1f-3ca8e46b9392 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:44:36.757953 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1242/2481] 10.222.0.33 () {58 vars in 1152 bytes} [Fri Jul 15 19:44:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:44:36.759143 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1518d5ab-cd53-402a-83f8-7e15bfade17b None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:44:36.770105 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1518d5ab-cd53-402a-83f8-7e15bfade17b None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:44:36.775356 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8692f71d-7c84-463b-9c4b-f61498495989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:44:36.776051 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8692f71d-7c84-463b-9c4b-f61498495989 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:36.776464 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8692f71d-7c84-463b-9c4b-f61498495989 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:36.776747 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8692f71d-7c84-463b-9c4b-f61498495989 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:36.777457 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8692f71d-7c84-463b-9c4b-f61498495989 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:44:36.780138 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8692f71d-7c84-463b-9c4b-f61498495989 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:44:36.904947 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1240/2482] 10.222.0.33 () {60 vars in 1152 bytes} [Fri Jul 15 19:44:36 2022] POST /identity/v3/projects => generated 373 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:44:36.909048 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd7eb95-5371-4dcd-8b1f-3ca8e46b9392 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:44:36.919015 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd7eb95-5371-4dcd-8b1f-3ca8e46b9392 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:44:36.974172 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51328fbb-61ae-42ed-9c91-c753b099d818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:44:36.974842 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51328fbb-61ae-42ed-9c91-c753b099d818 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:36.975174 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51328fbb-61ae-42ed-9c91-c753b099d818 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:36.975512 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51328fbb-61ae-42ed-9c91-c753b099d818 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:36.976139 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51328fbb-61ae-42ed-9c91-c753b099d818 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:44:36.978080 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51328fbb-61ae-42ed-9c91-c753b099d818 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:44:37.069094 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1243/2483] 10.222.0.33 () {60 vars in 1143 bytes} [Fri Jul 15 19:44:36 2022] POST /identity/v3/users => generated 287 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:44:37.073318 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8692f71d-7c84-463b-9c4b-f61498495989 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:44:37.084571 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8692f71d-7c84-463b-9c4b-f61498495989 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:44:37.089210 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f95687a-1460-44d1-83d9-ced199ba3dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:44:37.089961 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f95687a-1460-44d1-83d9-ced199ba3dfb admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:37.090249 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f95687a-1460-44d1-83d9-ced199ba3dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:37.090530 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f95687a-1460-44d1-83d9-ced199ba3dfb admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:37.091257 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f95687a-1460-44d1-83d9-ced199ba3dfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:44:37.092868 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f95687a-1460-44d1-83d9-ced199ba3dfb admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:44:37.103776 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1241/2484] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:44:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:44:37.107909 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51328fbb-61ae-42ed-9c91-c753b099d818 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:44:37.117789 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51328fbb-61ae-42ed-9c91-c753b099d818 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:44:37.123629 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ea3a7d-6f42-4ab6-bde8-1f73ccde83df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:44:37.124290 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ea3a7d-6f42-4ab6-bde8-1f73ccde83df admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:37.124574 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ea3a7d-6f42-4ab6-bde8-1f73ccde83df admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:37.124853 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ea3a7d-6f42-4ab6-bde8-1f73ccde83df admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:37.125576 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ea3a7d-6f42-4ab6-bde8-1f73ccde83df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:44:37.127202 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ea3a7d-6f42-4ab6-bde8-1f73ccde83df admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:44:37.137592 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1244/2485] 10.222.0.33 () {58 vars in 1121 bytes} [Fri Jul 15 19:44:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:44:37.142234 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f95687a-1460-44d1-83d9-ced199ba3dfb admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:44:37.152215 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f95687a-1460-44d1-83d9-ced199ba3dfb admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:44:37.156959 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f50e1a-c448-42b7-b589-f83170a84ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f47b71ea5a45ad92368f6b4130c36c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '392b3b5584be451c8c287eb751b4d7de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:44:37.157789 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f50e1a-c448-42b7-b589-f83170a84ee5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:37.158096 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f50e1a-c448-42b7-b589-f83170a84ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:37.158373 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f50e1a-c448-42b7-b589-f83170a84ee5 admin admin] PATH_INFO: `/v3/projects/4d6ba581f29f4bb4b3fb8c83bb44cd13/users/a8809f6c0df649e9a35366917f3697a1/roles/28763c6e70094116b3a4b2895895fa58` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:37.172903 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f50e1a-c448-42b7-b589-f83170a84ee5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d6ba581f29f4bb4b3fb8c83bb44cd13, user_id=a8809f6c0df649e9a35366917f3697a1, role_id=28763c6e70094116b3a4b2895895fa58)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:44:37.175930 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f50e1a-c448-42b7-b589-f83170a84ee5 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:44:37.257049 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1242/2486] 10.222.0.33 () {60 vars in 1482 bytes} [Fri Jul 15 19:44:37 2022] PUT /identity/v3/projects/4d6ba581f29f4bb4b3fb8c83bb44cd13/users/a8809f6c0df649e9a35366917f3697a1/roles/28763c6e70094116b3a4b2895895fa58 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:44:37.262220 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c0b0ac-b4a7-4d9e-b43d-1e5e0416c4bd None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:37.262534 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c0b0ac-b4a7-4d9e-b43d-1e5e0416c4bd None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:37.262812 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c0b0ac-b4a7-4d9e-b43d-1e5e0416c4bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:37.281602 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-53c0b0ac-b4a7-4d9e-b43d-1e5e0416c4bd None None] MFA Rules not processed for user `96f47b71ea5a45ad92368f6b4130c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:44:37.282876 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-53c0b0ac-b4a7-4d9e-b43d-1e5e0416c4bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:44:37.340142 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1245/2487] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:44:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:44:37.349724 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f50e1a-c448-42b7-b589-f83170a84ee5 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:44:37.351438 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-53f50e1a-c448-42b7-b589-f83170a84ee5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:44:37.357973 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:44:37.408242 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f50e1a-c448-42b7-b589-f83170a84ee5 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:44:37.411940 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eb3668-13cf-4695-af9a-e6dc60270ffd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:44:37.412593 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eb3668-13cf-4695-af9a-e6dc60270ffd service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:37.412878 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eb3668-13cf-4695-af9a-e6dc60270ffd service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:37.413155 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eb3668-13cf-4695-af9a-e6dc60270ffd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:37.423297 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27eb3668-13cf-4695-af9a-e6dc60270ffd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:44:37.429086 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27eb3668-13cf-4695-af9a-e6dc60270ffd service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:44:37.434237 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1243/2488] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:44:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:44:38.652391 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca4100d-f82b-45ac-95eb-827db05a5196 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:38.652859 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca4100d-f82b-45ac-95eb-827db05a5196 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:38.653017 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca4100d-f82b-45ac-95eb-827db05a5196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:38.674386 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:44:38.682570 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-1ca4100d-f82b-45ac-95eb-827db05a5196 None None] MFA Rules not processed for user `a8809f6c0df649e9a35366917f3697a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:44:38.683824 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-1ca4100d-f82b-45ac-95eb-827db05a5196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:44:38.751076 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1246/2489] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:44:38 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:44:38.761774 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eb3668-13cf-4695-af9a-e6dc60270ffd service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:44:38.763246 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-27eb3668-13cf-4695-af9a-e6dc60270ffd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:44:38.830704 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eb3668-13cf-4695-af9a-e6dc60270ffd service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:44:38.834292 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8ff15e-4667-4cb8-a584-8afb01186df5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86c2df3d378490ca7ea497a73c19d40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:44:38.834942 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8ff15e-4667-4cb8-a584-8afb01186df5 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:38.835226 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8ff15e-4667-4cb8-a584-8afb01186df5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:38.835504 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8ff15e-4667-4cb8-a584-8afb01186df5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:38.843867 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8ff15e-4667-4cb8-a584-8afb01186df5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:44:38.845638 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8ff15e-4667-4cb8-a584-8afb01186df5 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:44:38.854055 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1244/2490] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:44:38 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:44:41.694512 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c42a561-18b2-4625-8479-9332e8627fe3 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:41.695062 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c42a561-18b2-4625-8479-9332e8627fe3 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:41.695062 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c42a561-18b2-4625-8479-9332e8627fe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:41.714445 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-5c42a561-18b2-4625-8479-9332e8627fe3 None None] MFA Rules not processed for user `a8809f6c0df649e9a35366917f3697a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:44:41.715735 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-5c42a561-18b2-4625-8479-9332e8627fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:44:41.722576 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1247/2491] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:44:41 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:44:48.850061 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=50 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:44:51.708173 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=30 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:45:02.101644 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4786142e-f7f4-42e7-bf54-f5ae40b0f9c9 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:45:02.102265 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4786142e-f7f4-42e7-bf54-f5ae40b0f9c9 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:45:02.102431 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4786142e-f7f4-42e7-bf54-f5ae40b0f9c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:45:02.132119 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-4786142e-f7f4-42e7-bf54-f5ae40b0f9c9 None None] MFA Rules not processed for user `3d893c14620b4a129842062f14a68a78`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:45:02.146370 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-4786142e-f7f4-42e7-bf54-f5ae40b0f9c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:45:02.199816 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1245/2492] 10.222.0.33 () {58 vars in 1009 bytes} [Fri Jul 15 19:45:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:45:02.689536 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c42a561-18b2-4625-8479-9332e8627fe3 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:45:02.691007 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-5c42a561-18b2-4625-8479-9332e8627fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:45:02.764381 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c42a561-18b2-4625-8479-9332e8627fe3 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:45:02.776922 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f04c74c-efd3-45d8-8514-62be86f528a8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef76f7fe39a3499aaeebf85c176804f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:45:02.776922 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f04c74c-efd3-45d8-8514-62be86f528a8 service swift] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:45:02.776922 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f04c74c-efd3-45d8-8514-62be86f528a8 service swift] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:45:02.776922 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f04c74c-efd3-45d8-8514-62be86f528a8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:45:02.782419 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f04c74c-efd3-45d8-8514-62be86f528a8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:45:02.784120 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f04c74c-efd3-45d8-8514-62be86f528a8 service swift] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:45:02.802852 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1248/2493] 10.222.0.33 () {60 vars in 1384 bytes} [Fri Jul 15 19:45:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 19:45:03.503064 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4786142e-f7f4-42e7-bf54-f5ae40b0f9c9 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:45:03.505163 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-4786142e-f7f4-42e7-bf54-f5ae40b0f9c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:45:03.592198 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4786142e-f7f4-42e7-bf54-f5ae40b0f9c9 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:45:03.599152 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0537e2b1-1c5a-4d34-aa5c-e3bd409faad8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52a0fce231ba4785a685ac9898d07503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:45:03.600085 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0537e2b1-1c5a-4d34-aa5c-e3bd409faad8 service placement] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:45:03.600368 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0537e2b1-1c5a-4d34-aa5c-e3bd409faad8 service placement] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:45:03.600647 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0537e2b1-1c5a-4d34-aa5c-e3bd409faad8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:45:03.602711 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0537e2b1-1c5a-4d34-aa5c-e3bd409faad8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:45:03.615689 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0537e2b1-1c5a-4d34-aa5c-e3bd409faad8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:45:03.617410 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0537e2b1-1c5a-4d34-aa5c-e3bd409faad8 service placement] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:45:03.637944 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1246/2494] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:45:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:45:05.372858 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83a83ff-387b-4f6c-a57d-f3e71ce689af None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:45:05.373786 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83a83ff-387b-4f6c-a57d-f3e71ce689af None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:45:05.374251 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83a83ff-387b-4f6c-a57d-f3e71ce689af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:45:05.393673 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-b83a83ff-387b-4f6c-a57d-f3e71ce689af None None] MFA Rules not processed for user `a8809f6c0df649e9a35366917f3697a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:45:05.395460 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-b83a83ff-387b-4f6c-a57d-f3e71ce689af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:45:05.403478 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1249/2495] 10.222.0.33 () {58 vars in 957 bytes} [Fri Jul 15 19:45:05 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:45:13.625564 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=64 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:45:15.387908 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=35 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:45:26.623414 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6476df2-6b0c-43b4-8287-7f6f663660cd None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:45:26.623909 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6476df2-6b0c-43b4-8287-7f6f663660cd None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:45:26.624011 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6476df2-6b0c-43b4-8287-7f6f663660cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:45:26.641862 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-b6476df2-6b0c-43b4-8287-7f6f663660cd None None] MFA Rules not processed for user `3d893c14620b4a129842062f14a68a78`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:45:26.643071 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b6476df2-6b0c-43b4-8287-7f6f663660cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:45:26.650760 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1247/2496] 10.222.0.33 () {58 vars in 1009 bytes} [Fri Jul 15 19:45:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:45:26.663340 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83a83ff-387b-4f6c-a57d-f3e71ce689af None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:45:26.676008 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83a83ff-387b-4f6c-a57d-f3e71ce689af None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:45:26.681143 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4681a02c-6ee8-4db4-a86f-a139b1a0591b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef76f7fe39a3499aaeebf85c176804f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:45:26.681817 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4681a02c-6ee8-4db4-a86f-a139b1a0591b service swift] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:45:26.682104 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4681a02c-6ee8-4db4-a86f-a139b1a0591b service swift] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:45:26.682454 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4681a02c-6ee8-4db4-a86f-a139b1a0591b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:45:26.691745 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4681a02c-6ee8-4db4-a86f-a139b1a0591b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:45:26.693330 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4681a02c-6ee8-4db4-a86f-a139b1a0591b service swift] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:45:26.701663 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1250/2497] 10.222.0.33 () {60 vars in 1384 bytes} [Fri Jul 15 19:45:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 19:45:30.983985 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6476df2-6b0c-43b4-8287-7f6f663660cd None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:45:30.985564 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b6476df2-6b0c-43b4-8287-7f6f663660cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:45:30.999351 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6476df2-6b0c-43b4-8287-7f6f663660cd None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:45:31.002961 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847826f2-892c-4146-8ff1-291259a0bf67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:45:31.012415 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847826f2-892c-4146-8ff1-291259a0bf67 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:45:31.012415 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847826f2-892c-4146-8ff1-291259a0bf67 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:45:31.012415 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847826f2-892c-4146-8ff1-291259a0bf67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:45:31.012415 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-847826f2-892c-4146-8ff1-291259a0bf67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:45:31.097702 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847826f2-892c-4146-8ff1-291259a0bf67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:45:31.099179 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847826f2-892c-4146-8ff1-291259a0bf67 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:45:31.120540 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1248/2498] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:45:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 19:45:32.352556 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4681a02c-6ee8-4db4-a86f-a139b1a0591b service swift] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:45:32.362887 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4681a02c-6ee8-4db4-a86f-a139b1a0591b service swift] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:45:32.366838 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8817ab0c-7a50-4422-ad55-7a36a67b4a55 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86c2df3d378490ca7ea497a73c19d40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:45:32.367341 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8817ab0c-7a50-4422-ad55-7a36a67b4a55 service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:45:32.367981 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8817ab0c-7a50-4422-ad55-7a36a67b4a55 service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:45:32.368459 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8817ab0c-7a50-4422-ad55-7a36a67b4a55 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:45:32.370865 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-8817ab0c-7a50-4422-ad55-7a36a67b4a55 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:45:32.387816 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8817ab0c-7a50-4422-ad55-7a36a67b4a55 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:45:32.389358 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8817ab0c-7a50-4422-ad55-7a36a67b4a55 service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:45:32.404678 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1251/2499] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:45:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:45:41.117784 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=46 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:45:42.398435 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=12 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:45:42.907450 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847826f2-892c-4146-8ff1-291259a0bf67 service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:45:42.908818 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-847826f2-892c-4146-8ff1-291259a0bf67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:45:42.923801 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847826f2-892c-4146-8ff1-291259a0bf67 service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:45:42.934896 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6afcdb-ae69-4029-8581-b668c214c163 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7673a3b5c0324e07a9535ac70ed7a491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:45:42.934896 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6afcdb-ae69-4029-8581-b668c214c163 service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:45:42.934896 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6afcdb-ae69-4029-8581-b668c214c163 service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:45:42.934896 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6afcdb-ae69-4029-8581-b668c214c163 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:45:42.935232 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-2e6afcdb-ae69-4029-8581-b668c214c163 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:45:43.090128 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6afcdb-ae69-4029-8581-b668c214c163 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:45:43.093356 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6afcdb-ae69-4029-8581-b668c214c163 service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:45:43.123956 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1249/2500] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:45:42 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:45:48.260678 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8817ab0c-7a50-4422-ad55-7a36a67b4a55 service cinder] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:45:48.262088 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-8817ab0c-7a50-4422-ad55-7a36a67b4a55 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:45:48.276231 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8817ab0c-7a50-4422-ad55-7a36a67b4a55 service cinder] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:45:48.280651 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66249de6-964b-4f73-9b89-94ad322f54e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86c2df3d378490ca7ea497a73c19d40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:45:48.281271 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66249de6-964b-4f73-9b89-94ad322f54e2 service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:45:48.281548 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66249de6-964b-4f73-9b89-94ad322f54e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:45:48.281846 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66249de6-964b-4f73-9b89-94ad322f54e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:45:48.284082 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-66249de6-964b-4f73-9b89-94ad322f54e2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:45:48.364970 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66249de6-964b-4f73-9b89-94ad322f54e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:45:48.367114 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66249de6-964b-4f73-9b89-94ad322f54e2 service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:45:48.386681 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1252/2501] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:45:48 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:45:50.015049 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd491ffa-8794-44f2-b873-331e7271bdab None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:45:50.015532 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd491ffa-8794-44f2-b873-331e7271bdab None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:45:50.015642 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd491ffa-8794-44f2-b873-331e7271bdab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:45:50.033693 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-cd491ffa-8794-44f2-b873-331e7271bdab None None] MFA Rules not processed for user `3d893c14620b4a129842062f14a68a78`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:45:50.039312 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-cd491ffa-8794-44f2-b873-331e7271bdab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:45:50.042220 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1250/2502] 10.222.0.33 () {58 vars in 1009 bytes} [Fri Jul 15 19:45:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:45:50.054351 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66249de6-964b-4f73-9b89-94ad322f54e2 service cinder] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:45:50.066056 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66249de6-964b-4f73-9b89-94ad322f54e2 service cinder] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:45:50.070959 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2849e7-a70b-4133-8d75-24dce0bfb460 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef76f7fe39a3499aaeebf85c176804f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:45:50.071599 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2849e7-a70b-4133-8d75-24dce0bfb460 service swift] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:45:50.071881 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2849e7-a70b-4133-8d75-24dce0bfb460 service swift] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:45:50.072155 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2849e7-a70b-4133-8d75-24dce0bfb460 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:45:50.080576 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2849e7-a70b-4133-8d75-24dce0bfb460 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:45:50.082186 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2849e7-a70b-4133-8d75-24dce0bfb460 service swift] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:45:50.090070 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1253/2503] 10.222.0.33 () {60 vars in 1384 bytes} [Fri Jul 15 19:45:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 19:45:50.772245 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca31f0ca-beab-4210-8cce-490986edd6f1 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:45:50.772245 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca31f0ca-beab-4210-8cce-490986edd6f1 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:45:50.772245 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca31f0ca-beab-4210-8cce-490986edd6f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:45:50.794939 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-ca31f0ca-beab-4210-8cce-490986edd6f1 None None] MFA Rules not processed for user `3d893c14620b4a129842062f14a68a78`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:45:50.796720 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-ca31f0ca-beab-4210-8cce-490986edd6f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:45:50.807261 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1251/2504] 10.222.0.33 () {58 vars in 967 bytes} [Fri Jul 15 19:45:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:45:50.834677 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2849e7-a70b-4133-8d75-24dce0bfb460 service swift] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:45:50.848623 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2849e7-a70b-4133-8d75-24dce0bfb460 service swift] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:45:50.854315 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a79145-2526-4d64-9e8f-535833a7a271 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef76f7fe39a3499aaeebf85c176804f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:45:50.855084 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a79145-2526-4d64-9e8f-535833a7a271 service swift] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:45:50.855457 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a79145-2526-4d64-9e8f-535833a7a271 service swift] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:45:50.855975 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a79145-2526-4d64-9e8f-535833a7a271 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:45:50.869543 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a79145-2526-4d64-9e8f-535833a7a271 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:45:50.871571 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a79145-2526-4d64-9e8f-535833a7a271 service swift] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:45:50.881795 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1254/2505] 10.222.0.33 () {60 vars in 1384 bytes} [Fri Jul 15 19:45:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 19:46:00.787253 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=47 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:46:00.877699 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=53 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:46:01.221215 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca31f0ca-beab-4210-8cce-490986edd6f1 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:46:01.235695 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca31f0ca-beab-4210-8cce-490986edd6f1 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:46:01.239449 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a8f369-1adc-413f-b1c8-754de1843753 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86c2df3d378490ca7ea497a73c19d40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:46:01.243301 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a8f369-1adc-413f-b1c8-754de1843753 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:46:01.243792 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a8f369-1adc-413f-b1c8-754de1843753 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:46:01.244091 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a8f369-1adc-413f-b1c8-754de1843753 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:46:01.247071 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e3a8f369-1adc-413f-b1c8-754de1843753 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:46:01.338408 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a8f369-1adc-413f-b1c8-754de1843753 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:46:01.340095 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a8f369-1adc-413f-b1c8-754de1843753 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:46:01.352378 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1252/2506] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:46:01 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:46:11.346289 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=28 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:46:51.728266 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a79145-2526-4d64-9e8f-535833a7a271 service swift] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:46:51.738383 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a79145-2526-4d64-9e8f-535833a7a271 service swift] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:46:51.742002 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529c966e-55f4-4667-a7b7-b2b6c9d507ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:46:51.742670 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529c966e-55f4-4667-a7b7-b2b6c9d507ef service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:46:51.742954 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529c966e-55f4-4667-a7b7-b2b6c9d507ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:46:51.743388 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529c966e-55f4-4667-a7b7-b2b6c9d507ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:46:51.745537 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-529c966e-55f4-4667-a7b7-b2b6c9d507ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:46:51.757491 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529c966e-55f4-4667-a7b7-b2b6c9d507ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:46:51.758960 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529c966e-55f4-4667-a7b7-b2b6c9d507ef service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:46:51.767670 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1255/2507] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:46:51 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:47:01.763751 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=6 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:49:38.882016 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a8f369-1adc-413f-b1c8-754de1843753 service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:49:38.890721 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a8f369-1adc-413f-b1c8-754de1843753 service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:49:38.894091 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de15c42-3cfe-4c5a-b395-3e0dcb4eeda3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86c2df3d378490ca7ea497a73c19d40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:49:38.894730 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de15c42-3cfe-4c5a-b395-3e0dcb4eeda3 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:49:38.895008 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de15c42-3cfe-4c5a-b395-3e0dcb4eeda3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:49:38.895281 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de15c42-3cfe-4c5a-b395-3e0dcb4eeda3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:49:38.904115 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de15c42-3cfe-4c5a-b395-3e0dcb4eeda3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:49:38.905534 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de15c42-3cfe-4c5a-b395-3e0dcb4eeda3 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:49:38.914272 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1253/2508] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:49:38 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:49:48.910308 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=6 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:50:45.430547 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529c966e-55f4-4667-a7b7-b2b6c9d507ef service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:50:45.431960 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-529c966e-55f4-4667-a7b7-b2b6c9d507ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:50:45.447174 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529c966e-55f4-4667-a7b7-b2b6c9d507ef service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:50:45.452434 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b7fe7d-4ca4-4004-b592-f92e8dc67b69 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52a0fce231ba4785a685ac9898d07503', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:50:45.453082 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b7fe7d-4ca4-4004-b592-f92e8dc67b69 service placement] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:50:45.453367 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b7fe7d-4ca4-4004-b592-f92e8dc67b69 service placement] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:50:45.453646 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b7fe7d-4ca4-4004-b592-f92e8dc67b69 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:50:45.462385 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b7fe7d-4ca4-4004-b592-f92e8dc67b69 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:50:45.463971 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b7fe7d-4ca4-4004-b592-f92e8dc67b69 service placement] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:50:45.473849 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1256/2509] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:50:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:50:55.469624 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=6 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:50:55.689076 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de15c42-3cfe-4c5a-b395-3e0dcb4eeda3 service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:50:55.698194 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de15c42-3cfe-4c5a-b395-3e0dcb4eeda3 service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:50:55.701782 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc57cdf8-e434-4c8c-86c1-1a4668d9418a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c86c2df3d378490ca7ea497a73c19d40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:50:55.702414 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc57cdf8-e434-4c8c-86c1-1a4668d9418a service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:50:55.702689 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc57cdf8-e434-4c8c-86c1-1a4668d9418a service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:50:55.702959 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc57cdf8-e434-4c8c-86c1-1a4668d9418a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:50:55.712116 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc57cdf8-e434-4c8c-86c1-1a4668d9418a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:50:55.713532 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc57cdf8-e434-4c8c-86c1-1a4668d9418a service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:50:55.722715 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1254/2510] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:50:55 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:51:05.718258 dv-cinder-799871-34-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=6 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:52:50.138059 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b7fe7d-4ca4-4004-b592-f92e8dc67b69 service placement] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:52:50.147348 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b7fe7d-4ca4-4004-b592-f92e8dc67b69 service placement] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:52:50.150982 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3652ba5c-e2cb-43a0-87fa-59f778861b9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2a283d7d3594e37943bda4529d0a775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a756ccaef14cb49b26b0d68dd0b466', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:52:50.151614 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3652ba5c-e2cb-43a0-87fa-59f778861b9e service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:52:50.151896 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3652ba5c-e2cb-43a0-87fa-59f778861b9e service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:52:50.152173 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3652ba5c-e2cb-43a0-87fa-59f778861b9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:52:50.159903 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3652ba5c-e2cb-43a0-87fa-59f778861b9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:52:50.161334 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3652ba5c-e2cb-43a0-87fa-59f778861b9e service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:52:50.169300 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1257/2511] 10.222.0.33 () {60 vars in 1365 bytes} [Fri Jul 15 19:52:50 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 32 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:53:00.165790 dv-cinder-799871-34-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=6 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}